MSDTC memory leakage

More
2 years 5 months ago #13818 by miodragj
Hello,
we discovered memory leakage (Constant memory growth.) inside Distributed Transaction Coordinator (msdtc.exe) process.

Periodically we set some field to some values using this method from mssql server side:
exec('UPDATE wms.public.erp_request_master_data SET wms_importstatus = 1 WHERE request_id=170581 and document_id=3424623 and depositor=''###''') at PGNP_ERP_MData

We noticed in PGNP Profiler that before our updates(s) pgnp driver adds this line :

START TRANSACTION ISOLATION LEVEL READ COMMITTED

and after our update(s) are also added these two lines:

IAsyncPG::RequestExec(PrepareExec)
IAsyncPG::RequestExec(CommitExec)

Before starting our sw solution we restarted Distributed Transaction Coordinator (msdtc.exe) process.
Memory consumption of msdtc.exe immediately after restarting Distributed Transaction Coordinatorprocess is around 3-4 Mb.
Everything is fine during next few updates. Updates were periodically executed. Every 30sec.
At some point (Few update batches.), we noticed that there is a delay between 'START TRANSACTION ...' and first 'UPDATE ...' lines. Always very close to 5 sec.
After that situation, msdtc's memory consumption (Private Bytes / Process Explorer / Sysinternals) started to grow each time when any update is executed. The growth is around 20-50Mb per each batch of updates. It doesn't matter whether there is only one or many update statements.

The only solution so far was to manualy restart Distributed Transaction Coordinator (msdtc.exe) process, but that is of course unacceptable long term solution.

Anyone so far experienced this annoyance ?

SQL Server is : Windows Server 2012 R2 Standard (64-bit) 8Gb RAM
Database is : Microsoft SQL Server Standard (11.0.5058) (64-bit)

Regards,
Miodrag
Attachments:

Please Log in or Create an account to join the conversation.

More
2 years 5 months ago #13821 by Moderator
Thank you for reporting the issue! We are working on the fix.

Please Log in or Create an account to join the conversation.

More
2 years 4 months ago #13833 by dvasdekis
Hi guys,

I'm encountering what I think is the same issue. I'm using SSIS with the Greenplum driver (server edition), and after about 1.5M rows with a narrow table, I get an 'out of memory' issue. I'm running it in the SSIS designer, and I can confirm that it's the SSIS debug host process that's swelling in memory usage:

After 30 minutes:


After 60 minutes:


Any ETA on a fix?

Please Log in or Create an account to join the conversation.

More
2 years 3 months ago #13839 by Moderator
The leak issue with import into Greenplum could be different. Would you please send additional info to our support: connection string used, type of import used (legacy bulk, or fastload), trace captured in PGNP Profiler.

As for original issue, we made some changes but the fix is not ready for release yet. We continue work on it.

Please Log in or Create an account to join the conversation.

More
2 years 3 months ago #13845 by dvasdekis
Hi guys,

I figured it out - turns out the memory leak occurs when we load into a table with a default value for a column, where we don't populate that value.

It's easy to work around - we just generate that value either in a SSIS Derived Column, or by transferring the data to a staging table, and populating the core table with SQL on Greenplum. Took a bit to work out though.

Would you still like the PGNP Profiler trace? If so, how do I capture that?

Cheers!

Please Log in or Create an account to join the conversation.

More
2 years 3 months ago #13846 by Moderator
Instead of the profiler trace, could you e-mail us the table DDL with the default column? If possible, a test package would help as well. We will try to reproduce the issue.

If you want to capture a trace, launch PGNP Profiler "as Administrator", and click on the blue triangle icon in the toolbar (Capture Ctrl+E). Perform tests, and save the trace by clicking the Save (Ctrl+S) button. Corresponding menu items are available too. Press Alt key in order to show main menu.

Please Log in or Create an account to join the conversation.

Time to create page: 0.144 seconds
Powered by Kunena Forum