Difference Between Unique Rows

priyankach · March 20, 2020, 9:45am

What is difference between Unique Rows and Unique Rows(Hashset)?

madhavi.bade · March 20, 2020, 9:56am

Hi,
Both steps removes duplicate rows and leaves only unique row occurrences.

For Unique Rows, input stream should be sorted; otherwise, only consecutive double rows are evaluated correctly.
The Unique Rows (HashSet) step tracks duplicates in memory and does not require a sorted input to process duplicate rows.