Secure Impersonation: Removing a User From LDAP Results in Permission Denied Errors When Cleaning Up Their HDFS Artifacts

Problem

Users have been removed from LDAP, but their respective objects in Datameer remained. Upon reaching their retention policy, permission denied errors are seen in the conductor log when Housekeeping Service attempts to clean up these orphaned folders.

Cause

This occurs because in a Secure Impersonation environment, Datameer attempts to impersonate the owner of these artifacts when Housekeeping is working to remove them. Since the user is no longer available in LDAP, impersonation fails resulting in the permission denied errors.

Solution

As a workaround, perform the following steps:

  • Run the following MySQL query:
    SELECT uri FROM data WHERE status =1;

This query will output a list of artifacts that have been marked for deletion by Datameer

  • Manually remove the listed objects and their subfolders from HDFS. In the event of a large list, a simple shell script is advisable.
  • Execute the following query in MySQL after script has returned successfully:
    UPDATE data SET status = 2 WHERE status = 1;

This will tell Datameer that the artifacts have been successfully deleted.