Extensions and reporting

Edit Subject

Hello Pawel,
I am sorry, but I do not understand what kind of extension do you need. Can you describe it further?

I am not sure what exactly do you mean by generating the report automatically, but what may help you is to run the profiling job from batch file. You can do that and you can schedule running the batch file by any windows scheduler or cron if you want to do that on Linux.
My colleague already described how to use the DQ Analyzer in batch in question how to run DQ Analyzer on Linux.
What do you need is to use the runcif.bat in /runtime/bin folder. You have to create a plan file in DQ Analyzer and run the runcif.bat file with a plan filename and path as a parameter. It will create the profiling output without opening the DQ Analyzer GUI and is schedulable as a job, however if you need to view the results, you are limited to opening the profile in the DQ Analyzer GUI.
You may need to edit the runcif.bat file to make it work on your OS.

Feel free to ask any further questions if you need help with it.

Hello,

I need to do analysis and generate report automatically and this should be made by any scheduler. I mean, I need to have report in e.g. HTML format every day. If it is possible, that's great. But if not, is it possible to make some plugin which can do this?

Thank you for help.

Hello,
you may want to try profile2xml.bat batch file, which takes profile file as a parameter and exports it to an xml format file.

We do not support batch export to html, only from the GUI, but it may be possible for you to read the exported xml file to the html page or generate the web page using XSL transformation from the xml file.

Hello,

Converting from .profile to .xml works fine, but I have problem with runcif.bat. It looks like this:

D:\>"%DQC_HOME%\bin\runcif.bat" "D:\Program Files (x86)\Ataccama DQ Analyzer 7\output_files\DimCustomer4.plan"
Using java at: "C:\Program Files\Java\jdk1.6.0_26\bin\java.exe"
14.07.2011 10:04:45 [INFO] Using following licences:
14.07.2011 10:04:45 [INFO] D:\Program Files (x86)\Ataccama DQ Analyzer 7\runtime\license_keys\license.plf
14.07.2011 10:04:45 [INFO] Using DQ Analyzer configuration file D:\Program Files (x86)\Ataccama DQ Analyzer 7\output_files\DimCustomer4.plan
14.07.2011 10:04:45 [INFO] Runtime configuration file not supplied.
14.07.2011 10:04:45 [FATAL] Data source 'DB' specified by property dataSource must exist. [Profiler(Profiling, offending property path=/dataSource)]
14.07.2011 10:04:45 [FATAL] Data source 'DB' specified by property dataSource Name must exist. [DimCustomer(Jdbc Reader, offending property path=/dataSourceName)]
14.07.2011 10:04:45 [FATAL] There were 0 validation warnings and 2 validation errors in the plan.
14.07.2011 10:04:45 [FATAL] Unable to use the config file provided, exiting.

In GUI works fine.

Hello,
sorry, I didn't get that you are profiling DB table and doing drill-through as well. What do you need is to specify the data source in a configuration file called runtime configuration. This file is automaticly created when you run the job from GUI.
You will have to edit the runcif.bat file, where do you need to specify the runtime configuration file by adding -runtimeConfig attribute of the main class:
call "%~dp0\run_java.bat" %JAVA_OPTS% com.ataccama.dqc.processor.bin.CifProcessor -runtimeConfig runtimeCfg.xml %*

In the runtimeCfg.xml file, you need to specify the DB connection by JDBC URL, driver class name and logging credentials. For example for default Derby database, the runtimeCfg.xml file looks like this:

<?xml version='1.0' encoding='UTF-8'?>

<runtimeconfig>  

  <dataSources>    

    <dataSource driverclass="org.apache.derby.jdbc.ClientDriver" name="DB" user="demo" password="crypted:DES:+7qv5S1MTxpUJRv6UoOcSx3lYzyUh0QDto5HhuV6Izg=" url="jdbc:derby://localhost:1538/demo;create=true"/>

  </dataSources>

</runtimeconfig>

You'll get the url and driverclass from the GUI (I do not know what database do you use, if you use the Derby delivered with DQ Analyzer or something yours), name is the name of your datasource.
I do not think it is necessary to use drill-through in Profiling step in your case, because the drill-through results are not saved in the XML export.

Hello,

my database is Miscrosoft SQL Server 2008 R2.

What I did:

1. I edited the runcif.bat file:


call "%~dp0\run_java.bat" %JAVA_OPTS% com.ataccama.dqc.processor.bin.CifProcessor -runtimeConfig "D:\Program Files (x86)\Ataccama DQ Analyzer 7\runtime\bin\runtimeCfg.xml" %*


2. I created the runtimeCfg.xml file in \runtime\bin folder:


<?xml version='1.0' encoding='UTF-8'?>
<runtimeconfig>
<dataSources>
<dataSource driverclass="com.microsoft.sqlserver.jdbc.SQLServerDriver" name="DB" user="root" password="root" url="jdbc:sqlserver://localhost:1433;databaseName=AdventureWorksDW2008R2"/>
</dataSources>
</runtimeconfig>


I'm not sure if the driver class is good, because I couldn't find it in GUI.

I have this information in console:

Using java at: "C:\Program Files\Java\jdk1.6.0_26\bin\java.exe"
14.07.2011 12:55:40 [INFO] Using following licences:
14.07.2011 12:55:40 [INFO] D:\Program Files (x86)\Ataccama DQ Analyzer 7\runtime\license_keys\license.plf
14.07.2011 12:55:40 [INFO] Using DQ Analyzer configuration file D:\Program Files (x86)\Ataccama DQ Analyzer 7\output_files\DimCustomer4.plan
14.07.2011 12:55:40 [INFO] Using runtime configuration file D:\Program Files (x86)\Ataccama DQ Analyzer 7\runtime\bin\runtimeCfg.xml
14.07.2011 12:55:40 [FATAL] There are errors in your runtime configuration. Stopping runtime.

Hello,
could you please check, if you have the XML header on the first line? It seems like the forum formated my message that it looks like, there is an empty line at start, what fails in XML parser.

Hello,

I have wrong driver class, but I don't know which I should use. Can you help me?

This is text from console:

Using java at: "C:\Program Files\Java\jdk1.6.0_26\bin\java.exe"
14.07.2011 15:01:51 [INFO] Using following licences:
14.07.2011 15:01:51 [INFO] D:\Program Files (x86)\Ataccama DQ Analyzer 7\ru
ntime\license_keys\license.plf
14.07.2011 15:01:51 [INFO] Using DQ Analyzer configuration file D:\Program F
iles (x86)\Ataccama DQ Analyzer 7\output_files\DimCustomer4.plan
14.07.2011 15:01:51 [INFO] Using runtime configuration file D:\Program Files
(x86)\Ataccama DQ Analyzer 7\runtime\bin\runtimeCfg.xml
14.07.2011 15:01:51 [FATAL] Data source definition error.
java.lang.IllegalStateException: Data source 'DB' is not correctly configured: U
nable to find class 'com.microsoft.sqlserver.jdbc.SQLServerDriver' : com.microso
ft.sqlserver.jdbc.SQLServerDriver
at com.ataccama.dqc.processor.bin.config.DataSourceConfig.validateDriver
Class(DataSourceConfig.java:130)
at com.ataccama.dqc.processor.bin.config.DataSourceConfig.validate(DataS
ourceConfig.java:106)
at com.ataccama.dqc.processor.bin.config.RuntimeConfiguration.validate(R
untimeConfiguration.java:129)
at com.ataccama.dqc.processor.bin.CifProcessor.execute(CifProcessor.java
:263)
at com.ataccama.dqc.processor.bin.CifProcessor.main(CifProcessor.java:14
8)

Hello,
I suppose you have right class, but you may need to copy the sqljdbc4.jar from the runtime\lib\jdbc\MS SQL to runtime\lib, so it is included in the classpath of the batch file.

Hello,

It works now. Thank you very much. But I have another question. Is there any kind of API which could help me to develop plugins to DQ Analyzer?

Hello,
there is no API at the moment. What you may try to use are Eclipse plugins within DQ Analyzer, but nothing to call the runtime.

Could you tell me, what feature do you exactly need? It may be solveable in a some way or we may develop it in the future.

Hello,

is it possible to add my own analysis or something like this, which can be done by GUI?