Cloudera Navigator - Datameer

2 followers
0
Avatar

Hi,

We just set up cloudera navigator on our sandbox environment. I'm curious to know if it's possible to use navigator to view data lineage of jobs run within datameer.

Example:

1) I have a workbook with data

2) I do groupby, groupcount transformations, and add a filter

3) I execute the job

 

I'd like to see the lineage in cloudera navigator showing the transformations (groupby, groupcount, filter). Is this even possible?

 

Thanks!

Nikhil

Nikhil Srinidhi

1 comment

  • Avatar
    Joel Stewart

    The Datameer jobs are compiled in such a way that the individual functions that occur in your workbook definitions may not execute in the same way. Datameer optimizes the job based on the scope of the entire work that is expected. 

    You'll be able to view the same information with Datameer jobs as you would with any other YARN application, but you will not likely be able to see the individual functions along the way due to the compilation that takes place first. 

    0
Please sign in to leave a comment.