Class AvroTrevniRecordWriterBase<K,V,T>

java.lang.Object
org.apache.hadoop.mapreduce.RecordWriter<K,V>
org.apache.trevni.avro.mapreduce.AvroTrevniRecordWriterBase<K,V,T>
Type Parameters:
K - The type of key the record writer should generate.
V - The type of value the record wrtier should generate.
T - The type of the entries within the Trevni container file being written.
Direct Known Subclasses:
AvroTrevniKeyRecordWriter, AvroTrevniKeyValueRecordWriter

public abstract class AvroTrevniRecordWriterBase<K,V,T> extends RecordWriter<K,V>
Abstract base class for RecordWriters that writes Trevni container files.
  • Field Details

    • EXT

      public static final String EXT
      trevni file extension
      See Also:
    • META_PREFIX

      public static final String META_PREFIX
      prefix of job configs that we care about
      See Also:
    • part

      protected int part
      Counter that increments as new trevni files are create because the current file has exceeded the block size
    • writer

      protected AvroColumnWriter<T> writer
      Trevni file writer
    • schema

      protected org.apache.avro.Schema schema
      Provided avro schema from the context
    • meta

      protected org.apache.trevni.ColumnFileMetaData meta
      meta data to be stored in the output file.
  • Constructor Details

    • AvroTrevniRecordWriterBase

      public AvroTrevniRecordWriterBase(TaskAttemptContext context) throws IOException
      Constructor.
      Parameters:
      context - The TaskAttempContext to supply the writer with information form the job configuration
      Throws:
      IOException
  • Method Details