Teradata Bulk Load Utilities


(sivaraman gangadharan) #1

Hello ,
I would like to know if there is way we can use teradata bulk insert utilities instead of JDBC writer where the Batch Size=3000 records is the maximum . Its making the process much slower when writing millions of records.

please let me know if someone has used it already or documentation leading to use of Teradata Bulk load utilities.

Thanks
Siva


(Maksim Zhelyazkov) #2

Hello Siva,

The Ataccama Data Quality Center supports a number of databases, one of which is the Teradata database. All supported databases are listed in our documentation.

The Ataccama IDE comes with bundled database drivers. The Teradata driver was already installed automatically when you started the IDE for the first time. You can create a new database connection in the IDE the same way as for any other database and pick Teradata as a database type.

If you have any issues setting up the connection, you can open a support ticket and we will do our best to help you.

Kind Regards,

Maksim


(sivaraman gangadharan) #3

Hi Maksim ,
JDBC writer i mentioned is using Teradata drivers which comes with installation and it works fine for smaller subset of records. But when we try to load millions of records it becomes slow due to batch size limitation of 3000 rows . Is there a way to use Teradata bulk load utilities to improve the processing time ?

Thanks
Siva


(Maksim Zhelyazkov) #4

Hello Siva,

Teradata is one of our recommended databases, but we do not support it and we cannot test a solution for you.

However, the Teradata JDBC driver supports a number of techniques to maximize the performance of the database. You can find documentation for them in the Teradata documentation. I suggest trying JDBC FastLoad, by adding TYPE=FASTLOAD to the jdbc connection string as explained HERE.
Please, keep in mind that this is a third-party source, which we have not tested. We cannot guarantee if the information there is up to date and would work as described.

I hope this was helpful to you.

Kind Regards,
Maksim