KMS Kerberos Exception for Tez Jobs: Failed to find any Kerberos tgt

Problem

When executing a Datameer job in the Tez execution framework, the following stacktrace is generated when accessing data in an HDFS Encryption Zone: 

Caused by: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:489)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:776)
        at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
        at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1395)
        at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1497)
        at org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:1482)
        at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:451)
        at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:444)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:444)
        at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:387)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:852) 

Cause

This is a known issue affecting Tez - TEZ-2963

Solution

To work around this issue, add the following custom property to the Datameer Hadoop Cluster configuration: 

tez.dag.recovery.enabled=false

If required, contact Datameer technical support for more information.