How to Configure the Number of Mappers and Reducers


Go beyond Finding the optimal split size / split count and learn how to configure the number of mapper and reducers.


The property das.splitting.max-split-count applies to mappers only. 

As Datameer Custom Properties outlines, it is possible to configure the number of reducers. If you want to enforce a specific reduce count, set the property das.job.reduce.tasks to greater then zero (0).

If the reducer count should be adapted based on the split-count (map-task count), set the property


Be aware that setting either property can influence the performance impact on job execution.