Welcome to LEAD Support Forum Login | Register | Faq  

    LEAD Support Forum
  Resource to find answers and post technical questions about LEAD products.
Search    
   

Re: POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Started by artem at 06-24-2008 19:45. Topic has 3 replies.

Print Search « Previous Thread Next Thread »
  06-24-2008, 19:45
artem is not online. Last active: 6/27/2008 3:26:35 PM artem

Not Ranked
Joined on 06-25-2008
Posts 1
POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Reply Quote
Medical DICOM toolkit v 15.

While working with typical medical dataset that includes
about 5000 individual DICOM files I stumbled on the
problem of very problematic performance during the
construction of DICOMDIR (LDicomDir) object.

There are 2 ways to build the LDicomDir:
InsertDicomDS and InsertDicomFile. Performance problem
is present for both of them.

What I observe is that after inserting 1000 files or so, the time
of inserting new file increases greatly. What i see is at least
a quadratic increase :
  InsertionTime = O(N*N)
where N is the number of files already inserted.
 However, it should be theoretically possible to achieve a log(N) complexity,
if not linear one.

Example: on a high-end dual core PC, building DICOMDIR
object takes about 190 sec (for 5000 files). This is measured
when the files are memory files (to exclude HD performance
from consideration). For comparison, if i do the same dataset
but in groups of ~50 files (each group goes to a separate
DICOMDIR object) it only takes 20 sec! It seems to me like an
unacceptable situation.

Can something be done right now till this is addressed in future versions?
Is there any way to combine several LDicomDir objects into one larger
one? Plan would be to construct separate DICOMDIR for every SERIES
and then merge them.

Btw, are functions LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Thread-safe? I.e. it would be nice to use all cores to build the DICOMDIR.

Building DICOMDIR object is a very frequent procedure in our workflow
and is very important to us.

Thank you in advance,
Artem Mikheev
NYU School of medicine, Radiology


   Report 
  06-25-2008, 14:29
Adnan Ismail is not online. Last active: 9/4/2008 2:25:00 PM Adnan Ismail



Top 10 Posts
Joined on 07-31-2006
Posts 1,150
Re: POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Reply Quote
Does your program keep all datasets loaded in memory during the process? If the memory consumption increases continuously, Windows itself becomes slow in providing the application with needed memory.
If you don't think memory usage is related, let me know and I will run some tests here.

About thread safety, our DICOM C++ Class Library functions are generally thread-safe. However, if the bottleneck is caused by disk access or memory allocation, splitting the operation into different threads might add complexity without improving performance.
Adnan Ismail
LEADTOOLS Technical Support

   Report 
  06-27-2008, 10:51
ArtemMikheev is not online. Last active: 6/27/2008 3:35:04 PM ArtemMikheev

Not Ranked
Joined on 03-13-2007
Posts 3
Re: POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Reply Quote
Thank you for prompt reply.

1. I assume that I dont keep loaded datasets in memory. I allocate the LDicomDS object once and keep reusing it
by loading only DICOM headers (!!! 0 flag in LoadDS function).
Here is the C++ code:

LDicomDS* ds = new LDicomDS());

for ( int i=0; i<NumFiles; i++ )
     if ( ds->LoadDS( Name[i],0 )==DICOM_SUCCESS )
            Dir->InsertDicomDS( *ds,Name[i]  );

delete ds;

2. On thread safety, I was surprised but the LoadDS part is actually very quick here, it only takes
about 17 sec or so on 5000 files. The rest of 190 sec goes to InsertDicomDS function.
So do you think it is worth trying to parallelize the cycle from 2)  ?

3. Details: those 5000 files are about 620MB combined with majority of files  ~150K.
They are spread over 96 folders.



Thanks once again,
Artem Mikheev
NYU School of medicine, Radiology

   Report 
  06-29-2008, 12:37
Adnan Ismail is not online. Last active: 9/4/2008 2:25:00 PM Adnan Ismail



Top 10 Posts
Joined on 07-31-2006
Posts 1,150
Re: POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile
Reply Quote
Artem,
I got similar results when I tested with our C++ Class Lib Dicom DIR demo. I think the reason is related to searching the DIR information for patient ID and other information before inserting every new data set. As the DIR grows, the search time increases for each file. I can submit a report to our engineers about this to ask them if there is a way to optimize it. To do that please send an email to support@leadtools.com from a valid email address and mention this post.
Adnan Ismail
LEADTOOLS Technical Support

   Report 
Post
LEAD Support Fo... » Developer » DICOM/Medical I... » Re: POOR performance of LDicomDir::InsertDicomDS and LDicomDir::InsertDicomFile

Powered by Community Server, by Telligent Systems