Loading Profile...
Help get this topic noticed by sharing it on
Twitter,
Facebook, or email.
Twitter,
Facebook, or email.
I’m
frustrated
Cannot get exporting into an HTML file to work...
Hi there. I would like to export the analysis to an html file. I know it's possible as well as to xml. The problem I have is that when I press Export all that the app does is showing me a pop-up box saying it's exporting but giving me no choice as to whether I want to use html or xml. I don't even know what file it's exporting to... Can anyone help me with this? Thank you.
-
Hello,
you have probably missed the settings, which are not visible at the first screen of the export wizard.
Once you press Export, the Export Profile wizard opens. On the first screen you choose Data to export. Then you need to press Next (not Finish) and you get to Select Analyses screen where you choose the data to export. When you press Next again, you get to Output File screen, where you can specify the format and output filename.
-
dariusz.spiewak October 26, 2010 10:28Martin, when I click Export there is no wizard opening. Just a plain pop-up saying it's gathering information and then, after several minutes, it just quits, disappears. Could it be due to the size of the analysis file (.profile) that is 2.45 GB in size?EditDeleteRemove
-
-
dariusz.spiewak October 26, 2010 10:33One more thing... When I try Export with a smaller .profile file, it does give me what you are saying it should. But for the big one it displays: Progress Information: Operation in progress... and after a couple of minutes it just quits.EditDeleteRemove
-
-
-
-
-
I see, yes, it might be caused by the huge amount of data. It tries to create the tree of all the data and probably runs out of memory. May I ask you how many columns do you profile?
I'll look for a possible solution, but I am afraid the export fails for such amount of data.-
I have 20 columns but the number of rows is well above 5,000,000. Is there no way to increase the amount o memory allocated to the process?
-
-
Martin, I have just run an analysis on a small subset of this data (still 20 columns, though) of 417 rows and I get the same issue. The .profile file is now 15 MB. What do you think is going on?
-
-
-
-
-
Yes, I also thought it might be caused by something else when trying to reproduce your problem on my own.
I would like to ask you for some more help. Can you, please, open Error log, by opening Menu/Show View/Other and choosing error log from the list of views in that dialog? You can also filter it by writing error in the text field. Error log should open next to the Console view and it should contain exception which happened when the export fails. It may give us some advice about what is going on in your case.-
Would that help?
eclipse.buildId=M20080911-1700
java.version=1.5.0_17
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -os win32 -ws win32 -arch x86
Error
Tue Oct 26 11:38:58 BST 2010
Error occurred during status handling
java.lang.OutOfMemoryError: Java heap space
at com.sun.org.apache.xerces.internal.dom.AttributeMap.setNamedItem(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.DeferredElementNSImpl.synchronizeData(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.ElementNSImpl.getNamespaceURI(Unknown Source)
at cz.adastra.cif.commons.util.xml.XmlDomUtil.getChildElementsByNS(XmlDomUtil.java:255)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeArray(XmlBeanWriter.java:421)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:358)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:397)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeArray(XmlBeanWriter.java:459)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:358)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:283)
at cz.adastra.cif.commons.util.reflect.BeanUtil.setUpBean(BeanUtil.java:109)
at com.ataccama.profiling.results.storage.ResultBeanUtil.readBean(ResultBeanUtil.java:53)
at com.ataccama.profiling.results.model.RollUpItem.getResult(RollUpItem.java:73)
at cz.adastra.cif.gui.ui.profiler.viewer.drilldown.DrillDownHandler.A(Unknown Source)
at cz.adastra.cif.gui.ui.profiler.viewer.drilldown.DrillDownHandler.isEnabled(Unknown Source)
at org.eclipse.core.commands.Command.isEnabled(Command.java:833)
at org.eclipse.core.commands.Command.setHandler(Command.java:965)
at org.eclipse.ui.internal.handlers.HandlerAuthority.updateCommand(HandlerAuthority.java:457)
at org.eclipse.ui.internal.handlers.HandlerAuthority.processChangedCommands(HandlerAuthority.java:631)
at org.eclipse.ui.internal.handlers.HandlerAuthority.access$1(HandlerAuthority.java:610)
at org.eclipse.ui.internal.handlers.HandlerAuthority$1.propertyChange(HandlerAuthority.java:175)
at org.eclipse.ui.internal.services.EvaluationAuthority$1.run(EvaluationAuthority.java:252)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
at org.eclipse.ui.internal.services.EvaluationAuthority.fireServiceChange(EvaluationAuthority.java:246)
at org.eclipse.ui.internal.services.EvaluationAuthority.endSourceChange(EvaluationAuthority.java:197)
at org.eclipse.ui.internal.services.EvaluationAuthority.sourceChanged(EvaluationAuthority.java:135)
at org.eclipse.ui.internal.services.ExpressionAuthority.sourceChanged(ExpressionAuthority.java:310)
at org.eclipse.ui.internal.services.ExpressionAuthority.sourceChanged(ExpressionAuthority.java:289)
at org.eclipse.ui.AbstractSourceProvider.fireSourceChanged(AbstractSourceProvider.java:99)
at org.eclipse.ui.internal.services.ActiveShellSourceProvider.access$13(ActiveShellSourceProvider.java:1)
at org.eclipse.ui.internal.services.ActiveShellSourceProvider$3.handleEvent(ActiveShellSourceProvider.java:302)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84) -
-
And there's another one (different kind, I suppose):
eclipse.buildId=M20080911-1700
java.version=1.5.0_17
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -os win32 -ws win32 -arch x86
Error
Tue Oct 26 12:00:49 BST 2010
Unhandled event loop exception
cz.adastra.cif.commons.util.reflect.WrappingRuntimeException: Work finished with exception
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService.A(Unknown Source)
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService.run(Unknown Source)
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService.run(Unknown Source)
at cz.adastra.cif.gui.ui.profiler.viewer.export.ProfileExportWizard.initExport(Unknown Source)
at cz.adastra.cif.gui.ui.profiler.viewer.export.ExportHandler.execute(Unknown Source)
at cz.adastra.cif.gui.ui.profiler.viewer.AbstractResultTable$1.widgetSelected(Unknown Source)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:228)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1003)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3823)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3422)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2382)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2346)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2198)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:493)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:288)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:488)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at com.ataccama.dqc.gui.application.AtcmApplication.start(Unknown Source)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:193)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
Caused by: java.lang.reflect.InvocationTargetException
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:415)
at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:507)
at org.eclipse.ui.internal.progress.ProgressMonitorJobsDialog.run(ProgressMonitorJobsDialog.java:275)
at org.eclipse.ui.internal.progress.ProgressManager$3.run(ProgressManager.java:977)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
at org.eclipse.ui.internal.progress.ProgressManager.busyCursorWhile(ProgressManager.java:1012)
at org.eclipse.ui.internal.progress.ProgressManager.busyCursorWhile(ProgressManager.java:987)
at org.eclipse.ui.internal.progress.ProgressManager.run(ProgressManager.java:1195)
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService$ExceptionHoldingRunnableWithSignal.run(Unknown Source)
at cz.adastra.cif.gui.commons.swt.DisplayTools.runAsyncInDisplay(Unknown Source)
at cz.adastra.cif.gui.commons.swt.DisplayTools.runAsyncInStandardDisplay(Unknown Source)
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService.run(Unknown Source)
... 29 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.getNodeObject(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.DeferredDocumentImpl.synchronizeChildren(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.DeferredElementNSImpl.synchronizeChildren(Unknown Source)
at com.sun.org.apache.xerces.internal.dom.ParentNode.getChildNodes(Unknown Source)
at cz.adastra.cif.commons.util.xml.XmlDomUtil.getChildElements(XmlDomUtil.java:186)
at cz.adastra.cif.commons.util.xml.XmlDomUtil.getChildElementsByNS(XmlDomUtil.java:250)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeArray(XmlBeanWriter.java:421)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:358)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:397)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeArray(XmlBeanWriter.java:459)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:358)
at cz.adastra.cif.commons.beanwriter.XmlBeanWriter.deserializeProperties(XmlBeanWriter.java:283)
at cz.adastra.cif.commons.util.reflect.BeanUtil.setUpBean(BeanUtil.java:109)
at com.ataccama.profiling.results.storage.ResultBeanUtil.readBean(ResultBeanUtil.java:53)
at com.ataccama.profiling.results.model.RollUpItem.getResult(RollUpItem.java:73)
at com.ataccama.profiling.results.model.RollUpItem.setExport(RollUpItem.java:144)
at com.ataccama.profiling.results.model.RollUp.setExports(RollUp.java:102)
at com.ataccama.profiling.results.model.RollUpItem.setExport(RollUpItem.java:147)
at com.ataccama.profiling.results.model.RollUp.setExports(RollUp.java:102)
at com.ataccama.profiling.results.model.ProfileInput.setExport(ProfileInput.java:161)
at com.ataccama.profiling.results.model.ProfileResultRoot.setExport(ProfileResultRoot.java:129)
at cz.adastra.cif.gui.ui.profiler.viewer.export.ProfileExportWizard$4.run(Unknown Source)
at cz.adastra.cif.gui.commons.swt.utils.PurityProgressService$1.run(Unknown Source)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:121) -
-
-
-
-
Hello,
both problems are the same as you may see by the line "java.lang.OutOfMemoryError: Java heap space".
There is a way how to add more heap space in memory for the DQ Analyzer, but I am not currently sure it will help you. You can edit dqa.ini file in the installation directory and change the parameter -Xmx256m to -Xmx512m or -Xmx768m, but as I say I am afraid it won't help you for your huge file and if it does, the export will take long time.
Thanks for the report, I'll ask our development if they have any other advices or if we can do anything else about it.
May I ask you if you setup any roll-ups to be done during the analysis? Analysis of 5 mils of records on 20 columns doesn't usually create such a huge file.-
Yes, I do have a roll-up. But I wouldn't say it's huge - there are 7 categories. The real concern is that even for 417 records containing 20 columns, the Analyzer is extremely slow and doesn't let me export into anything. When I click on a category it takes like several minutes to load the data, and after the 4. load - it freezes. By the way, Java is notoriously slow, as you might know.
-
-
Ok, now, the smaller set will export to xml pretty fast. I'm trying to export into zipped html. It looks like the Profiler is doing something... Will let you know how it went. And yes, I have changed the heap space to -Xmx768m.
-
-
-
-
-
Yes, the roll-up has been designed for smaller amount of enumerated values to distinguish different systems, countries or genders, it might work as a tool to have views on the data from different dimensions, but it might not work well - I suppose using OLAP tool in this case would be better.
On the other hand we might do something about this. The problem might be caused by the UI component on the first screen of Export wizard as it draws a node for every combination of roll-up value and column name (it might be a tree consisting of milions of nodes, Windows will even have a problem with drawing it). We can design either an command line application or different "Direct to html wizard" for cases like yours (on the other hand we should redesign the HTML page as well, as having such a tree in the menu won't be usable).
Thanks for the report, we'll think about it and how to make the case possible.-
Martin. One last thing about the Frequency Analysis. Would it not be much, MUCH, better if that could be done so that instead of giving the lower, upper limit and the number of steps, the app determined the boundaries itself and used a formula for the number of bins (one I can think of is, say, sqrt(n), where n is the number of observations, that is, records; this could also be something like a*log(n)+b - many statistical books report such formulas) or one could enter the number of bins? What I am trying to propose here is the automatic detection of the boundaries, which would free the analyst from having to determine them themselves (as I have had to do). I think that would be an extremely neat solution. Best. Darek.
-
-
-
-
EMPLOYEE
