Hi All,
We are facing performance issue while loading and exporting data in MDM. We have around 30 million records. We are loading data in chunks and initial few execution were very quick. After processing 20M we are observing severe degrade in performance. Now even for 10K records the job is taking 8-10 hours for both load and export. We have applied parallelism wherever its possible. Is there any other thing which we need to consider/implement which will enhance the performance?
TIA.
Best answer by ivysakh
Hi
A thorough investigation is necessary to pinpoint potential bottlenecks in the process. By examining the load operation and individual MDM processes, you can identify the specific step that is taking too long to complete, such as matching, merging, or change detection. Once identified, you can analyze the configuration of that step in detail to uncover any issues. For instance, complex match rules with fuzzy logic can impact performance as data volume grows.
To get started, please refer to this online documentation for guidance on performance tuning in MDM:
https://docs.ataccama.com/mdm/14.5.x/performance-tuning/performance-tuning.html
If you need additional help, feel free to contact the support team by opening a case.
+3Hi
A thorough investigation is necessary to pinpoint potential bottlenecks in the process. By examining the load operation and individual MDM processes, you can identify the specific step that is taking too long to complete, such as matching, merging, or change detection. Once identified, you can analyze the configuration of that step in detail to uncover any issues. For instance, complex match rules with fuzzy logic can impact performance as data volume grows.
To get started, please refer to this online documentation for guidance on performance tuning in MDM:
https://docs.ataccama.com/mdm/14.5.x/performance-tuning/performance-tuning.html
If you need additional help, feel free to contact the support team by opening a case.
+3Hi ā
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
