How to Monitor Datameer Core Directories for Thresholds


How to monitor Datameer core directories for certain thresholds? 


Instead of using CLI commands 

hadoop fs -du -s -h /user/datameer
hadoop fs -du -h /user/datameer

and review the output, there should be more advanced solutions available. E.g. one could setup cron jobs to generate daily log files 

hadoop fs -du -h /user/datameer > /logs/hdfsusage.log.`date +"%Y-%m-%d"`


You are able to leverage quotas for specific directories, as per documentation HDFS Quotas Guide. This should assist with the needs, i.e. for monitoring space thresholds in the Datameer core directory.

Further Information 

You may also have a look into distribution specific documentation.