Skip to main content
Solved

ONE Expression - Trim

  • May 30, 2024
  • 6 replies
  • 115 views

JTH
Star Blazer L1
Forum|alt.badge.img+2
  • Star Blazer L1

Hi,

There are certain records that contain “white spaces” at the beginning and/or at the end. Ex: “    100137069       “, “    100017129       “, etc. We don't allow the use of white spaces and want to build a DQ rule to check the invalid records. Hence the advanced expression:  length(String_ATTRIBUTE_1) <> length(trim(String_ATTRIBUTE_1)). When I test the DQ rule it works as intended. I copy the above results and paste them in the test environment and it shows me the record is invalid. When I remove the white spaces it will considered valid. However, when I run the Monitoring Project based on the same records and DQ rule it gives me 100% validity. How is this possible? And maybe more importantly, how can I create a rule that will show the invalid records shown earlier?
 

 

Best answer by AKislyakov

Hi @JTH,

By default, Ataccama trims leading and trailing spaces for all columns. If you need to preserve them, you need to set following parameter:

-Dengine.trimStringValues=false (doc: JVM Configuration :: Ataccama ONE)

6 replies

Forum|alt.badge.img+2
  • Ataccamer
  • Answer
  • May 30, 2024

Hi @JTH,

By default, Ataccama trims leading and trailing spaces for all columns. If you need to preserve them, you need to set following parameter:

-Dengine.trimStringValues=false (doc: JVM Configuration :: Ataccama ONE)


jdordregter
Data Voyager
Forum|alt.badge.img+1
  • Data Voyager
  • May 30, 2024

😂 So in this case Ataccama increases the data quality before checking the data quality. 


Forum|alt.badge.img+1

Hello @JTH I’m closing this thread, please feel free to follow up here with any of your questions or create a new post 🙋‍♀️


Marcel-Jan
Star Blazer L2
Forum|alt.badge.img+1
  • Star Blazer L2
  • June 6, 2024

I still have some questions. I'm the person on @JTH 's team who implements the solution. And I don't understand where this -Dengine.trimStringValues property should be implemented. I read that it should be set in the Runtime Server. We have an on premise implementation of Ataccama One. And I don't think we have that running?

Normally when data is read I see it go passed the dpe, dpm or mmm-backend components. But they don't seem to have an engine related property.


JTH
Star Blazer L1
Forum|alt.badge.img+2
  • Author
  • Star Blazer L1
  • June 13, 2024

@AKislyakov @Cansu can any of you please follow up on @Marcel-Jan's comment so we can proceed on this issue further. We want to see what we can do with this solution 😊


Forum|alt.badge.img+2
  • Ataccamer
  • June 13, 2024

Hi @Marcel-Jan, @JTH 

For Ataccama ONE you need to set this property on DPE using plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.JAVA_OPTS
Like following:

plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.env.JAVA_OPTS=-Dengine.trimStringValues=false

Please note, that this parameter only works if plugin.executor-launch-model.ataccama.one.launch-type-properties.LOCAL.exec is configured (DPE Configuration :: Ataccama ONE)