MapBufferTooSmallException: Record too large for in-memory buffer
Recommendations under Container is Running Beyond Physical Memory Limits were followed and increasing Mapper and Reducer memory partitions to 16G did not improve the issue. The following error message is received:
task:org.apache.tez.runtime.library.common.sort.impl.ExternalSorter$MapBufferTooSmallException: Record too large for in-memory buffer. Exceeded buffer overflow limit, bufferOverflowRecursion=2, bufferList.size=1, blockSize=104857600
1. First add the following to the
Custom Properties of the job:
2. If the job still fails, remove the above property and add the following:
3. If execution also fails, attempt the job with both properties added.