Skip to main content

Is there a best approach to have LKPs updated via a webservice so that other web services can consume the updated lkp without restarting DQS?

 

I have Web Service ‘A’ which builds a dictionary file. 

I also have Web Service ‘B’ which uses the dictionary built by Web Service ‘A’.

I would like to run Web Service ‘A’ so that Web Service ‘B’ is able to immediately use the updated dictionary without restarting the DQS service. 

 

 

Currently when i run Web Service ‘ A’ I get a locked file issue:

26.07.2022 12:43:56 uFATAL]   2OnlineServicesComponent]LbuidTestDictionary.online]ebuidTestDictionary]abuidTestDictionary.comp] n40] Internal error occurred during run of the plan: java.nio.channels.OverlappingFileLockExceptionlSTEP Lookup BuildercLookup Builder]]
java.nio.channels.OverlappingFileLockException
    at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source)
    at sun.nio.ch.SharedFileLockTable.add(Unknown Source)
    at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source)
    at com.ataccama.dqc.io.storage.RafROStorage.open(RafROStorage.java:49)
    at com.ataccama.dqc.tasks.common.data.table.StandaloneTableBuilder.<init>(StandaloneTableBuilder.java:103)
    at com.ataccama.dqc.tasks.builders.LookupBuilder$TheInstance.runStandard(LookupBuilder.java:328)
    at com.ataccama.dqc.tasks.common.hadoop.MRStepInstanceBase.run(MRStepInstanceBase.java:32)
    at com.ataccama.dqc.processor.internal.runner.ComplexStepNode.runNode(ComplexStepNode.java:69)
    at com.ataccama.dqc.processor.internal.runner.RunnableNode.run(RunnableNode.java:28)
    at com.ataccama.dqc.commons.threads.AsyncExecutor$RunningTask.run(AsyncExecutor.java:131)
    at java.lang.Thread.run(Unknown Source)

 

 

any ideas or best approaches to do this will be appreciated. I have added the location of the LKP file to the 

‘versionedFile’ segement in the ServerConfig xml file and this get the same issue. 

Thanks

Hi Asif,
 

Does the .lkp file being used by web service B exist in the same location as the .lkp file being updated by web service A? If so, this might be the root cause of the overlapping file lock error message you are receiving.

There is the best approach method to refreshing .lkp files in production that are being used by other services. The approach involves having two separate locations for your .lkps files.

The first location ext_new/lkp/ is where the build plans would create the new lkp files. Whilst the second location ext/lkp/ is where other services would be pointed to. (web service X, please use the .lkp files in this location).

With a technical workflow (.ewf) the new .lkp files would be created, moved to the location of the existing .lkps and the virtual file system refreshed so all the services are aware of the .lkp files have changed on the file system.

There is an example of this process in the ONE Desktop application, under the Workflow Tutorials / 02_Run_Plans / 02_00_Build_Lookups.ewf which might be helpful for you.

I hope this helps, please let us know how to get on.
Have a great day.

Danny

Hi Danny,

Many thanks for your reply. Sorry for my delayed response to your update.

You suggestion of moving the file would work but the problem we do not have the workflow capability. We simply have DQS only. We do not have any other similar tools we can use on the machine where DQS resides.

I noticed that the file lock is enforce by the JVM. So i was able to configure the FileVersion in the serverConfig file and then create 2 DQS window services. One which updates the LKP and another which uses it. Its not ideal but it gets around the problem. 

Thank you.

Asif

Reply


Community Terms of Use & ConditionsCookie settings
ataccama
arrows
Lead your team  forward  OCT 24 / 9AM ET
Register now
×