Skip to main content
Solved

Not enough memory to run the plan file.

  • December 6, 2022
  • 9 replies
  • 276 views

Forum|alt.badge.img+4

I want to load around 832,032 records from Excel file to DB table using Plan. Plan is not executing.

It giving below error message :-

1)Not enough memory to run the plan file.

 

  1. java.lang.OutOfMemoryError: Java heap space
        at org.apache.commons.io.IOUtils.byteArray(IOUtils.java:338)
        at org.apache.commons.io.output.AbstractByteArrayOutputStream.needNewBuffer(AbstractByteArrayOutputStream.java:104)
        at org.apache.commons.io.output.UnsynchronizedByteArrayOutputStream.<init>(UnsynchronizedByteArrayOutputStream.java:51)
        at org.apache.poi.util.IOUtils.toByteArray(IOUtils.java:181)
        at org.apache.poi.util.IOUtils.toByteArray(IOUtils.java:153)
        at org.apache.poi.openxml4j.util.ZipArchiveFakeEntry.<init>(ZipArchiveFakeEntry.java:67)
        at org.apache.poi.openxml4j.util.ZipInputStreamZipEntrySource.<init>(ZipInputStreamZipEntrySource.java:98)
        at org.apache.poi.openxml4j.opc.ZipPackage.<init>(ZipPackage.java:132)
        at org.apache.poi.openxml4j.opc.OPCPackage.open(OPCPackage.java:312)
        at com.ataccama.dqc.io.msoffice.OpenXmlFile.open(OpenXmlFile.java:71)
        at com.ataccama.dqc.tasks.msoffice.excel.read.ExcelFileReaderInstance.runEventDriven(ExcelFileReaderInstance.java:196)
        at com.ataccama.dqc.tasks.msoffice.excel.read.ExcelFileReaderInstance.runStream(ExcelFileReaderInstance.java:119)
        at com.ataccama.dqc.tasks.msoffice.excel.read.ExcelFileReaderInstance.run(ExcelFileReaderInstance.java:97)
        at com.ataccama.dqc.processor.internal.runner.ComplexStepNode.runNode(ComplexStepNode.java:69)
        at com.ataccama.dqc.processor.internal.runner.RunnableNode.run(RunnableNode.java:28)
        at com.ataccama.dqc.commons.threads.AsyncExecutor$RunningTask.run(AsyncExecutor.java:135)
        at java.base/java.lang.Thread.run(Thread.java:834)

Anyone faced this situation? How to resolve this issue? Thanks in advance.

Best answer by AKislyakov

Hi everyone @Ojaswini @greglvaughan , 

Because of Support portal re-design the KB articles are indeed currently unavailable. Here is how to allocate more memory to a single plan:

 

  1. Click the drop down arrow beside run and select Run Configuration…
  2. From within the Runtimes tab, and in VM arguments, you can input the same variables for the VM that are added in the environment variable JAVA_OPTS.
  3. Set the Xmx variable, which configures to the maximum heap size reserved by the Java Virtual Machine. Follow the notation for passing this parameter, e.g., -Xmx2g stands for reserving 2 GB of memory as the maximum allocated for the plan. Notice that you can see a complete list of all the plans being opened and executed by DQC in the left pane, with their particular VM argument settings.

     

 

View original
Did this topic help you find an answer to your question?

9 replies

Forum|alt.badge.img+2
  • Ataccamer
  • 146 replies
  • December 6, 2022

Hi Ojaswini,

There is the Knowledge Base article on how to deal with the bigger plans. In your case I think the section Allocating Additional Memory for a Particular Plan Execution suits best.


Forum|alt.badge.img+4
  • Author
  • Star Blazer L1
  • 15 replies
  • December 6, 2022

Thank you very much.


Forum|alt.badge.img+4
  • Author
  • Star Blazer L1
  • 15 replies
  • December 7, 2022

Yes, that helped to run the plan. Basically, I had to change memory limit at all specified location in the given link. Thank you.


greglvaughan
Data Enthusiast
Forum|alt.badge.img
  • Data Enthusiast
  • 16 replies
  • March 10, 2024

HI,

Those links are redirecting to the new support root page.  Are the knowledge base articles still available?

 

thanks,

 

Greg

 


Forum|alt.badge.img+4
  • Author
  • Star Blazer L1
  • 15 replies
  • March 11, 2024

Hi,

 

You are right. They're not available now. Or maybe I don't the new links for those. Given links areonly redirecting to root page.

Thanks,

Ojaswini


Forum|alt.badge.img+2
  • Ataccamer
  • 146 replies
  • Answer
  • March 12, 2024

Hi everyone @Ojaswini @greglvaughan , 

Because of Support portal re-design the KB articles are indeed currently unavailable. Here is how to allocate more memory to a single plan:

 

  1. Click the drop down arrow beside run and select Run Configuration…
  2. From within the Runtimes tab, and in VM arguments, you can input the same variables for the VM that are added in the environment variable JAVA_OPTS.
  3. Set the Xmx variable, which configures to the maximum heap size reserved by the Java Virtual Machine. Follow the notation for passing this parameter, e.g., -Xmx2g stands for reserving 2 GB of memory as the maximum allocated for the plan. Notice that you can see a complete list of all the plans being opened and executed by DQC in the left pane, with their particular VM argument settings.

     

 


greglvaughan
Data Enthusiast
Forum|alt.badge.img
  • Data Enthusiast
  • 16 replies
  • March 13, 2024

HI,

 

Thanks so much! Is there a timeline for , or a way to access the old knowledge base articles.

 

Greg


Cansu
Community Manager
Forum|alt.badge.img+3
  • Community Manager
  • 625 replies
  • March 13, 2024

Hi @greglvaughan, you can find the versions from 13.9 and lower in PDF formats here. Our team is working to get them on the new portal at the moment, you can learn more about it here.


greglvaughan
Data Enthusiast
Forum|alt.badge.img
  • Data Enthusiast
  • 16 replies
  • March 14, 2024

Thanks @Cansu   I will take a look!  

All the best!

gV


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings