Skip to main content
Solved

Text File Writer: adding a datetime as suffix to an output file

  • November 29, 2023
  • 6 replies
  • 119 views
Albert de Ruiter
Star Blazer L2
Forum|alt.badge.img+4

Hey all,

I was wondering if you ever came accross the following requirement and has some tips maybe to implement it.

The Text File Writer step can be used to create for instance log files. For tracking or governance reasons it may be necessary to keep all log files. But because the output file name is fixed, each time the file will be overwritten whenever a new one is created. Of course it is possible to manually add a datetimestamp to the file name after its creation, but that is a rather cumbersome approach. So I would like to be able to add a datetimestamp to the file name automatically. 

Does anyone have a suggestion?

Kind regards,

Albert

Best answer by SamWrigley

Hi Albert, 

My approach to this is by adding the Alter Format step before the Text File Writer step.
For my purpose I use date and timeNow with the following expressions, if you want a datetime stamp you can just add the time format to the date expression.


In the Text File Writer step you can then set the filename using this syntax,

{date}/logfile_{timeNow}.txt

In this example it creates a new folder with todays date and adds the time to the logfile. ​​​

View original
Did this topic help you find an answer to your question?

6 replies

S
Space Explorer
Forum|alt.badge.img+2
  • SamWrigleySpace Explorer
  • Space Explorer
  • 5 replies
  • Answer
  • November 29, 2023

Hi Albert, 

My approach to this is by adding the Alter Format step before the Text File Writer step.
For my purpose I use date and timeNow with the following expressions, if you want a datetime stamp you can just add the time format to the date expression.


In the Text File Writer step you can then set the filename using this syntax,

{date}/logfile_{timeNow}.txt

In this example it creates a new folder with todays date and adds the time to the logfile. ​​​

Albert de Ruiter
C
Cansu
3 people like this
  • Albert de Ruiter
    Albert de Ruiter
  • C
    Charles Lenard
  • Cansu
    Cansu
Albert de Ruiter
Star Blazer L2
Forum|alt.badge.img+4

Hi Sam,

That is a really nice solution, thanks a lot 😀!

Regards,

Albert

S
Cansu
2 people like this
  • S
    SamWrigley
  • Cansu
    Cansu
Joeri
Data Pioneer
Forum|alt.badge.img+4
  • JoeriData Pioneer
  • Data Pioneer
  • 34 replies
  • November 28, 2024

Why is this not working in JDBC Writer, in calling a component, or other objects? Or is this functionality only available later than 14.5.1 ?

-- Joe
Albert de Ruiter
Star Blazer L2
Forum|alt.badge.img+4

Hi Joeri,

I used this approach first in version 13.9. 

Maybe the following is preventing you to write the date to a database. The date in above example has type string and that may not be sufficient to write the value to a date data type column in a table.

Kind regards,

Albert

Cansu
1 person likes this
  • Cansu
    Cansu
Joeri
Data Pioneer
Forum|alt.badge.img+4
  • JoeriData Pioneer
  • Data Pioneer
  • 34 replies
  • November 28, 2024

No, we are trying to put a STRING type data flow element in a field of Component parameter, or a Filename field in a JDBC writer or Parquet File writer, nothing works. I’ll escalate via Ataccama.

-- Joe
Albert de Ruiter
Star Blazer L2
Forum|alt.badge.img+4

OK good luck with your issues ​@Joeri 

Reply


Most Liked this week

Community Terms of Use & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings