Package com.exasol.parquetio.splitter
Class ParquetFileSplitter
java.lang.Object
com.exasol.parquetio.splitter.ParquetFileSplitter
- All Implemented Interfaces:
FileSplitter
A class that splits Parquet file into chunks of certain size.
Each chunk then contains one or more row group start and end positions in a Parquet file.
-
Constructor Summary
ConstructorsConstructorDescriptionParquetFileSplitter(org.apache.parquet.io.InputFile file) Creates a new instance ofParquetFileSplitter.ParquetFileSplitter(org.apache.parquet.io.InputFile file, long chunkSize) Creates a new instance ofParquetFileSplitter. -
Method Summary
Modifier and TypeMethodDescriptionprotected List<ChunkInterval>getRowGroupSplits(List<org.apache.parquet.hadoop.metadata.BlockMetaData> rowGroups) Returns row group splits given the file row groups.Gets file splits in the form ofstartandendintervals.
-
Constructor Details
-
ParquetFileSplitter
public ParquetFileSplitter(org.apache.parquet.io.InputFile file) Creates a new instance ofParquetFileSplitter. It uses default chunk size of64MBto split the file.- Parameters:
file- a Parquet file
-
ParquetFileSplitter
public ParquetFileSplitter(org.apache.parquet.io.InputFile file, long chunkSize) Creates a new instance ofParquetFileSplitter.- Parameters:
file- a Parquet filechunkSize- a chunk size in bytes
-
-
Method Details
-
getSplits
Description copied from interface:FileSplitterGets file splits in the form ofstartandendintervals.- Specified by:
getSplitsin interfaceFileSplitter- Returns:
- an array of intervals
-
getRowGroupSplits
protected List<ChunkInterval> getRowGroupSplits(List<org.apache.parquet.hadoop.metadata.BlockMetaData> rowGroups) Returns row group splits given the file row groups.- Parameters:
rowGroups- a list of file row groups- Returns:
- a list of
ChunkIntervals
-