Custom Function that performs lookup

Comments

5 comments

  • Gido

    In general, this is possible. With the SimpleFunctionType you will be able to take a number of arguments and return a result value.

    https://documentation.datameer.com/documentation/current/Tutorial02+-+Building+Custom+Functions

    Within the function you need to cover the part of connecting to a remote HBase in Java then.

    0
    Comment actions Permalink
  • Guhan Kashyap

    Thanks Gido. I'm still confused about what the runtime support is. Can I connect to any HBase, HDFS source? Can I connect to other sources like Vertica? Are there limitations to what servers I can connect to or can I pretty much use any Java functionality?

    Also - are there other restrictions on the functions such as side effect restrictions?

    0
    Comment actions Permalink
  • Gido

    In respect to your most recent question, if you could use pretty much any Java functionality, the answer is yes.

    About your your question what the runtime support is, I am not sure. Can you provide more context regarding this?

    0
    Comment actions Permalink
  • Guhan Kashyap

    In terms of context - I was thinking of something like using Spark SQL in the custom functions - can that be done?

    0
    Comment actions Permalink
  • Joel Stewart

    Guhan - I'd be cautious about implementing a custom function that requires feedback from an outside system. This may cause a lot of lag when using the function since the result must be computed and displayed on the sample data. This may provide a poor user experience.

    0
    Comment actions Permalink

Please sign in to leave a comment.