Dynamics GP – Identify who deleted a batch using the SQL transaction log

Introduction

On occasion a batch can be accidentally deleted from the system, and for one reason or another it would be interesting to find out which user did this. In this post I aim to show how this might be possible using some SQL functions to query the SQL transaction log. (I appreciate there are options available to prevent users deleting batches however I’m not covering this)

Pre-requisite and Considerations

One pre-requisite is that the SQL database must be in the FULL recovery model. This won’t work in the SIMPLE model as the transaction log information is lost at each checkpoint.

Another consideration you must make is that the process of posting a batch in Dynamics GP effectively deletes the batch header from the SY00500 table as well. Therefore, the result set from the query could include both genuine deletes, caused by posting batches, and deletes that have happened accidentally. However if you know a little detail about the batch that you wish to track like the type of batch that was deleted, and approximately the number of transactions in the batch, I’m hoping the result you get back from this query will assist you in making an educated guess as to if this is actually the deleted batch you were looking for.

The Theory

In SQL every INSERT\UPDATE\DELETE operation is first logged in the transaction log before being written to the database. Therefore, theoretically using a SQL function you can query the transaction log looking for DELETEs to tables and objects.

I mentioned earlier that the database must be in the FULL recovery model. To expand on this if the database is in the SIMPLE recovery model then after the INSERT\UPDATE\DELETE transaction commits, and a checkpoint occurs, that transaction log information is lost. Therefore you won’t be able to query the transaction log for deletes when the database is in SIMPLE recovery model.

However when the database is in the FULL recovery model, the log information isn’t lost until a LOG backup is performed. Therefore using a SQL function you can query the log looking for deletes.

Finally, all is not lost if you have performed a log backup since the delete occurred as you can use another SQL function to query the transaction log backup file that resides on disk. You just need to know the file name of the backup and its location on disk.

Use FN_DBLOG to query the transaction log

To demonstrate the process, I first created the following batch in Dynamics GP which contains one transaction.

I then deleted the batch from Dynamics GP and using the query below I can identify SQL transactions that include a delete on the batch headers table. (SY00500). Please note this needs running the company database.

SELECT [transaction name], 
       users.NAME, 
       [transaction id], 
       operation, 
       allocunitname, 
       [begin time], 
       [end time], 
       [transaction name] 
FROM   Fn_dblog(NULL, NULL) trnlog 
       LEFT JOIN sys.sysusers users 
              ON trnlog.[transaction sid] = users.sid 
WHERE  [transaction id] IN (SELECT [transaction id] 
                            FROM   Fn_dblog(NULL, NULL) 
                            WHERE  operation IN ( 'LOP_DELETE_ROWS' ) 
                                   AND allocunitname LIKE '%sy00500%')   

This gives the following results

As you can from the results above there’s a SQL Transaction that includes a deletion to the batch header table (SY00500) and associated objects (i.e. the indexes). This is SQL Transaction ID 0000-00045d05. As part of the same transaction you can see deletes to GL tables and objects therefore its clear this is a GL batch that was deleted. You can also see the user who deleted the batch in the “name” column.

Use FN_DUMP_DBLOG to query the transaction log

If a transaction log backup has occurred since the batch was deleted, you won’t be able to use the query above as the log will have been cleared. However you can query the SQL transaction log backup file directly using a different SQL function called FN_DUMP_DBLOG.

To test this I performed a transaction log backup and gave it a file name of C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\Backup\TWO18_3.trn and used the query below to look for the same delete I performed earlier. (please note in this query I had to change a few things as the object name didn’t work without joining to another system table)

SELECT [transaction name], 
       users.NAME, 
       [transaction id], 
       operation, 
       Object_name(part.object_id) AS ObjectName, 
       [begin time], 
       [end time], 
       [transaction name] 
FROM   Fn_dump_dblog (NULL, NULL, N'DISK', 1, 
N'C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\Backup\TWO18_3.trn' 
, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, 
DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT) trnlog 
LEFT JOIN sys.sysusers users 
       ON trnlog.[transaction sid] = users.sid 
LEFT JOIN sys.partitions part 
       ON trnlog.partitionid = part.partition_id 
WHERE  [transaction id] IN (SELECT [transaction id] 
                            FROM   Fn_dump_dblog (NULL, NULL, N'DISK', 1, 
N'C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\Backup\TWO18_3.trn' 
, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, 
DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT) trnlog 
LEFT JOIN sys.partitions part 
       ON trnlog.partitionid = part.partition_id 
 WHERE  operation IN ( 'LOP_DELETE_ROWS' ) 
        AND Object_name(part.object_id) = 'sy00500')   

This gave me the output below which is the same as the original output. Its just the data resides in the transaction log backup file now. (i.e. its the same SQL Transaction ID)

If you don’t find the delete you can just alter the backup file name and keep searching through your backups.

Conclusion

Therefore although not 100% ideal this is one method that can assist in identifying a user that has deleted a batch in Dynamics GP.

You can also use the same method to find out which user deleted other transactions you just need to alter the object name.

I’d also like to note that although I have tweaked the SQL scripts I’ve used in this post myself, I most certainly had to stand on the shoulders of several SQL giants along the way. I’m sure everyone is aware the SQL community is also awesome at sharing information to help others like me!

Thanks for reading

Dynamics GP – Clear Activity in Bank Management – Delete from CBEU1020

I recently had an issue where a user was stuck in a batch in the Bank Management module. You can usually run the “Clear Activity” option to resolve the issue however on this occasion it didn’t work. I found I had to manually delete a row in an SQL table to clear the lock and allow the user access to the batch.

The exact issue they received is as follows:

With this error message the first thing to try is to clear activity and although this didn’t work on this occasion I’ll detail the steps below.

Select “Tools > Routines > Financial > Bank Management > Clear Activity”

Select “Clear Activity”

Ensure there are no users in Dynamics GP and click “Yes” to the message below. (rather than asking everyone to log out I usually just ensure there’s nobody doing any Bank Management activities)

After doing this you will be prompted with the message below confirming activity is cleared

However I found I still couldn’t access the batch. On investigation there is still some activity for the batch in the CBEU1020 table which needed clearing. I therefore ran the SQL query below in the company database to delete the row. (replace TWO18 with the name of your company database)

I could then access the batch.

Thanks for reading

Dynamics NAV \ Business Central – Expected Costing with Purchasing Transactions

Background

Within Dynamics NAV \ Business Central you can switch Expected Cost Posting to G/L both ON and OFF via the option below in Inventory Setup.

In this post I’ve been playing with this feature to see how things work and how the various postings differ to Dynamics GP. I also take a look at how the Value Entries in Inventory play a pivotal role in this. I end by taking a closer look at the SQL tables involved and how things fit together.

Expected Costs in Dynamics GP

When you receive goods via a Shipment transaction in Dynamics GP a Purchase Accrual is automatically created to a General Ledger accrual account to record the expected cost in the General Ledger. This account is generally taken from the Creditor Card as per below:

Dynamics GP – Creditor Account Maintenance

The opposite debit entry is taken from the Inventory Item card as per below:

Dynamics GP – Item Account Maintenance

Let’s add a Purchase Order Shipment transaction in Dynamics GP and see this in action:

Dynamics GP – Receiving Transaction Entry

As you can see from the screen shot above, I’m receiving one Inventory item, and this has created an accrual distribution crediting the 000-2111-00 accrued purchases account I specified on the creditor card. The balancing debit side is to the Inventory code that we specified on the Inventory Item card.

Now let’s see what happens when we invoice the Shipment:

Dynamics GP – Purchase Invoice Entry

Just as expected the accrual is reversed via a Debit entry to the 000-2111-00 accrued purchases account and the accounts payable is credited. Therefore, the balance in the accrual account is now nil.

There’s no way to disable this behaviour in Dynamics GP. When you post a “Shipment” for some Inventory Items General Ledger entries are always created. (however you can prevent the entries posting through to the General Ledger via the Posting Setup)

Expected Costs in Dynamics NAV \ Business Central

Before we look at Expected Costing in Dynamics NAV \ Business Central we first have to take a step back and look at the various inventory entries that are created when you post inventory transactions.

When you post an inventory transaction in Dynamics NAV \ Business Central the system creates a minimum of two inventory entries: an Item Ledger Entry and a Value Entry. The Item Ledger Entry records the change in quantity and the Value Entry records the change in inventory values. For the purposes of this post we just need to know that when posting Purchase receipts Value Entries are created for Expected Costs, and when you post Purchase Invoices, Value Entries are created for Actual Costs, and Expected Costs are reversed.

Expected Cost Posting to G/L – Switched ON

Unlike Dynamics GP you can switch ON and OFF accrual postings in Dynamics NAV \ Business Central via the Expected Cost Posting to G/L option in Inventory Setup. When you switch Expected Cost Posting to G/L ON interim accounts are used to post the accrual and inventory entries for Purchase receipt transactions.

The equivalent Dynamics GP accrued purchases account is called Invt. Accrual Acc. (Interim) and is specified in the General Posting Setup window and is selected based on the Posting Groups used on the Item and Creditor. (see my previous post for more details on the posting groups). I’ve highlighted this below

Dynamics NAV \ Business Central – General Posting Setup

The Inventory code for the debit side of the transaction is taken from the Inventory Posting Group and again is based on the combination of posting groups used. I’ve highlighted this below

Dynamics NAV \ Business Central – Inventory Posting Setup

The key difference here is Dynamics GP doesn’t use an interim Inventory account whereas Dynamics NAV \ Business Central does.

In my Cronus demo data, the option Expected Cost Posting to G/L is currently switched ON so let’s see how this works when I create a Purchase Order for an Inventory item and receive it.

Here’s my Purchase Order and after clicking Post I’m going to choose receive so I receive the items into the Inventory:

When I view the item I can see this has created the following Value Entry (Number 454) which shows the Cost Amount (Expected) and Expected Cost Posted to G/L both populated.

If I highlight the Value Entry and click Navigate > General Ledger I can see the G/L Entries associated with the Value Entry

As expected, we can see the 5510 Accrual account is being credited and the debit entry is posting to the 2111 “Interim” Inventory code.

Now let’s invoice the purchase order and take a look at the G/L entries. First I click Post and select Invoice on the Purchase Order. (Incidentally if I were to select Receive and Invoice the system recognises I’ve already received the items. It doesn’t receive them again)

This has created the a new Value Entry (Number 455) . There’s a few things to note here. Firstly the Cost Amount (Expected) and the Expected Cost to G/L have been reversed. Secondly the Cost Amount (Actual) and Cost Posted to G/L have been populated.

Again if I highlight the Value Entry and choose Navigate > General Ledger we can the G/L Entries associated with this Value Entry.

As you can see the original entries created via the Purchase Receipt have been reversed by posting a debit to the 5510 Inventory accrual account and a credit to the 2111 Interim Inventory account. The system has then posted new entries to the 2210 Resale items inventory account (debit) and the direct cost applied account (credit). (for more info on the direct cost applied account see my previous post).

So that’s it. Although there are a few extra distributions to Dynamics GP everything makes sense. Its also apparent that the Value Entries have a direct relation to the G/L entries that are created.

Expected Costing to G/L – Switched OFF

Now let’s see what happens when we post a Purchase Receipt with the Expected Cost Posting to G/L switched OFF. First I switch the option OFF and then create and post the Purchase Receipt

This has created the following Value Entries (Number 456)

The key thing to note here is that although the Cost Amount (Expected) is populated the Expected Cost Posted to G/L isn’t. This means no G/L Entries have been created. To prove this click Navigate > General Ledger to view any G/L Entries

Let’s now invoice the Purchase Order and see what happens:

This has created the following Value Entry

This Value Entry records the Cost Amount (Expected) being reversed and the Cost Amount (Actual) and Cost Posted to G/L are also populated. Therefore we get the following G/L Entries

As expected no expected cost postings have been created or reversed for this transaction.

Bonus – G/L Item Ledger Relation and Post Value to G/L SQL tables

An unexpected bonus of writing this post was the chance to geek out on some of the Dynamics NAV tables. Unlike Dynamics GP, I don’t have much of a grasp of the SQL tables in Dynamics NAV however while going through the various scenario’s I was curious about how a couple of things worked which encouraged me to dig a little deeper.

The first thing was how I was able to drill down on the G/L Entries from the Value Entries screen? This meant there must be a direct or indirect relationship between the tables.

After some digging I found the link was via the G_L – Item Ledger Relation table. Therefore writing the SQL query below enabled me to join the G/L Entries and Value Entries table to see all the details for Value Entry Number 455

The next thing I was curious about was what would happen if the Expected Cost Posting to G/L was switched from OFF to ON when there were lots of Purchase Receipts that hadn’t yet been invoiced?

I found the answer to this question lay in the two prompts you receive when you switch Expected Cost Posting to G/L from OFF to ON (or ON to OFF). Below are the two messages you get when toggling the setting

Dynamics NAV \ Business Central – Inventory Setup

As per the first message it seems when you switch the option Expected Cost Posting to G/L ON the system determines if the Actual Costs for the Purchase Receipt have been posted and if not a record is written for that Value Entry to the Post Value Entry to G_L SQL table. This has a link back to the Value Entry so the system knows to create the Expected Cost interim postings for this Value Entry.

To show this in action, I switched the Expected Cost Posting to G/L option OFF and queried the SQL table:

SQL Query on Post Value Entry to GL Table

As per the image above the SQL table is currently blank.

I then created a Purchase Order and received it as per below:

Dynamics NAV \ Business Central – Purchase Order

After posting this I queried the Post Value Entry to G/L SQL table again to see if any new rows had been added and the table was still blank

SQL Query on Post Value Entry to GL Table

I then checked the Value Entry for my receipt and as per the screen shot below the Cost Amount (Expected) is populated but the Expected Cost Posted to G/L is blank. As this is only the Purchase Receipt the Cost Posted to G/L is also zero.

Dynamics NAV \ Business Central – Value Entries

I then went back to Inventory Setup and switched Expected Cost Posting to G/L back ON and clicked YES to the prompt and now when I check the Post Value Entry to G/L table its populated as per below

SQL Query on Post Value Entry to GL Table

After toggling the Expected Cost Posting to G/L option to ON the system has determined that this Value Entry has no G/L Entries for the Expected Costs and has inserted a record into the Post Value Entry to G_L table with a direct link back to the Value Entry that was created when I posted my receipt.

Now if I run the Post Inventory Cost to G/L batch job as instructed in the second message G/L entries are created for the purchase receipt, the SQL table is cleared, and the Value Entry is updated. See below:

The report output of the “Post Inventory Cost to G/L” shows entries have been created:

Dynamics NAV \ Business Central – Post Inventory Cost to G/L

Below are the expected cost General Ledger entries created to the interim accounts. (in my previous example these were created immediately because I had Post Expected Costs to G/L switched ON)

Dynamics NAV \ Business Central – General Ledger Entries

And finally the Expected Cost Posted to G/L field on the Value Entry has been updated to show the General Ledger entries have been created and posted.

Dynamics NAV \ Business Central – Value Entries

If I now check the Post Value Entry to G_L table in SQL I can see its been cleared.

Incidentally if I were to switch OFF Expected Cost Posting to G/L before running the Post Inventory Cost to G/L batch job the SQL row is removed from the Post Value Entry to G_L table.

Conclusion

In conclusion I find the way Dynamics GP deals with expected costs to be a much more conventional and simple approach however there’s no doubting that Dynamics NAV \ Business Central gives more flexibility.

Although I don’t know much about the inner workings of Dynamics NAV \ Business Central it also seems to me that G/L Entries are created based on the Value Entries.

In another post I hope to look at how Expected Costs work with Sales Shipments and Sales Invoices.

Thanks for reading.