java.lang.OutOfMemoryError


(Kalicharan Khetwal) #1

Hello Team,

I’m getting below two errors while performing profiling task.

Detail Error:
ava.lang.OutOfMemoryError
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:233)
at com.ataccama.dqc.io.data.store.PartitionedStreamFile$MInputStream.read(PartitionedStreamFile.java:132)
at com.ataccama.dqc.io.data.store.PartitionedStreamFile$MInputStream.read(PartitionedStreamFile.java:116)
at com.ataccama.dqc.commons.serialize.Serialize.read(Serialize.java:525)
at com.ataccama.dqc.commons.serialize.Serialize.deserializeBigLong(Serialize.java:114)
at com.ataccama.dqc.commons.serialize.SerializerFactory$6.deserialize(SerializerFactory.java:114)
at com.ataccama.dqc.io.sorting.Frequency$Item$ItemSerializer.deserialize(Frequency.java:248)
at com.ataccama.dqc.io.sorting.Frequency$Item$ItemSerializer.deserialize(Frequency.java:234)
at com.ataccama.dqc.io.data.store.FileObjectStore.read(FileObjectStore.java:61)
at com.ataccama.dqc.io.data.store.ObjectStore.read(ObjectStore.java:67)
at com.ataccama.dqc.tasks.profiling.common.GlobalRuntimeContext$SparingSorter.rewind(GlobalRuntimeContext.java:225)
at com.ataccama.dqc.io.sorting.Frequency.rewind(Frequency.java:166)
at com.ataccama.dqc.tasks.profiling.runtime.PdRuntime$PdProcess.computeValues(PdRuntime.java:385)
at com.ataccama.dqc.tasks.profiling.runtime.PdRuntime$PdProcessBase.finishChunk(PdRuntime.java:275)
at com.ataccama.dqc.tasks.profiling.runtime.CompositeProfilingRuntime$CompositeProfilingProcess.finishChunk(CompositeProfilingRuntime.java:117)
at com.ataccama.dqc.tasks.profiling.runtime.RollUpWrap.compute(RollUpWrap.java:319)
at com.ataccama.dqc.tasks.profiling.ProfilingAlgorithmInstance$InputRunnable.compute(ProfilingAlgorithmInstance.java:459)
at com.ataccama.dqc.tasks.profiling.ProfilingAlgorithmInstance$InputRunnable.run(ProfilingAlgorithmInstance.java:411)
at java.lang.Thread.run(Thread.java:745)

Thanks !


(Katrin Popova) #2

Hello Ramzi,

Please review this article for memory optimization.

Regards,
Katrin