Component Behavior


(Keith Isom) #1

Hello,

I some have questions regarding component steps.

When after a component is invoked, does the output data structure from the component become the current data structure on the connection, essentially deleting any prior structure?

From the documentation, the text reads:

Modifying the format inside the Component. Most steps preserve the format of incoming data; however some, such as Union, alter the data format and must be accounted for in the containing plan file.

In what sense does ‘must be accounted for’ mean?

Regards,
Keith


(Katrin Popova) #2

Hello Keith,

I think you may find this explanation regarding column mappings within components useful.

Regards,
Katrin


(Keith Isom) #3

Thank you for the link.

I was a little confused about the statement towards the end. It says: ‘[t]he auto-mapping happens because the data format inside the component changes’.

I now understand that this does not mean the format was changed by an Alter Format step.

It means that a Join, Representative Creator, Group Aggregator or Union is used in the component and therefore could possibly alter the data by turning a single data instance into potentially more than one instance, a Result Set if you will.

This has the dramatic impact of replacing the current data on the connector, and destroying previous columns that may be referenced later. And this change in format is simply due to any reference to Join, etc., at build/compile time, and not at run-time.

In my case, a JSON Call step causes the same behavior even without an offending JOIN, etc. I suspect that DQ is interpreting this step as potentially returning more than one data instance and therefore replaces the connector content. And this is despite the fact that my JSON Call returns only one record, ever.

My solution is to alter the calling component to always expect a Result Set. However it would be a convenient option to alter DQ’s behavior in the Component Step by a setting that indicates the result will always be a single data instance regardless of Joins, etc.