Skip to main content

Hey all,

I was wondering if you ever came accross the following requirement and has some tips maybe to implement it.

The Text File Writer step can be used to create for instance log files. For tracking or governance reasons it may be necessary to keep all log files. But because the output file name is fixed, each time the file will be overwritten whenever a new one is created. Of course it is possible to manually add a datetimestamp to the file name after its creation, but that is a rather cumbersome approach. So I would like to be able to add a datetimestamp to the file name automatically. 

Does anyone have a suggestion?

Kind regards,

Albert

Hi Albert, 

My approach to this is by adding the Alter Format step before the Text File Writer step.
For my purpose I use date and timeNow with the following expressions, if you want a datetime stamp you can just add the time format to the date expression.


In the Text File Writer step you can then set the filename using this syntax,

{date}/logfile_{timeNow}.txt

In this example it creates a new folder with todays date and adds the time to the logfile. ​​​

Hi Sam,

That is a really nice solution, thanks a lot 😀!

Regards,

Albert

Why is this not working in JDBC Writer, in calling a component, or other objects? Or is this functionality only available later than 14.5.1 ?

Hi Joeri,

I used this approach first in version 13.9. 

Maybe the following is preventing you to write the date to a database. The date in above example has type string and that may not be sufficient to write the value to a date data type column in a table.

Kind regards,

Albert

No, we are trying to put a STRING type data flow element in a field of Component parameter, or a Filename field in a JDBC writer or Parquet File writer, nothing works. I’ll escalate via Ataccama.

OK good luck with your issues ​@Joeri 

Reply


Community Terms of Use & ConditionsCookie settings