pegasus.mapreduce.common.cache

Class FileUtils

java.lang.Object

pegasus.mapreduce.common.cache.FileUtils

public class FileUtils
extends java.lang.Object

An utility class to manipulate files.

Constructor Summary

Constructors

Constructor and Description
FileUtils()

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	getLocalTemporaryPath(java.lang.String name, org.apache.hadoop.mapreduce.TaskAttemptContext context) Returns a temporary directory path.
static org.apache.hadoop.io.compress.CompressionCodec	getPreferredCompressionCodec(org.apache.hadoop.conf.Configuration conf) Returns the preferred compression codec.
static org.apache.hadoop.io.compress.CompressionCodec	getPreferredCompressionCodec(org.apache.hadoop.mapreduce.TaskAttemptContext context) Returns the preferred compression codec.
static java.io.DataInputStream	getTemporaryFileInputStream(org.apache.hadoop.fs.Path path, org.apache.hadoop.mapreduce.TaskAttemptContext context) Creates an input stream from the given path of a file.
static java.io.DataOutputStream	getTemporaryFileOutputStream(org.apache.hadoop.fs.Path path, org.apache.hadoop.mapreduce.TaskAttemptContext context) Creates an output stream to the given path of a file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileUtils

public FileUtils()

Method Detail

getLocalTemporaryPath

public static org.apache.hadoop.fs.Path getLocalTemporaryPath(java.lang.String name,
                                                              org.apache.hadoop.mapreduce.TaskAttemptContext context)

Returns a temporary directory path.

Parameters:

name - The name of directory

context - Hadoop context to create new directory

Returns:

A temporary directory path

getTemporaryFileInputStream

public static java.io.DataInputStream getTemporaryFileInputStream(org.apache.hadoop.fs.Path path,
                                                                  org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                           throws java.io.IOException

Creates an input stream from the given path of a file. This method assumes that the file is compressed by the preferred compression codec.

Parameters:

path - The path of a file

context - Hadoop context to open a file

Returns:

An input stream from the given file

Throws:

java.io.IOException

getTemporaryFileOutputStream

public static java.io.DataOutputStream getTemporaryFileOutputStream(org.apache.hadoop.fs.Path path,
                                                                    org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                             throws java.io.IOException

Creates an output stream to the given path of a file. This method compresses the values to be stored.

Parameters:

path - The path of a file

context - Hadoop context to create a file

Returns:

An output stream to the given file

Throws:

java.io.IOException

getPreferredCompressionCodec

public static org.apache.hadoop.io.compress.CompressionCodec getPreferredCompressionCodec(org.apache.hadoop.conf.Configuration conf)

Returns the preferred compression codec.

This method returns SnappyCodec object if it is available, otherwise DefaultCodec object is returned.

Parameters:

conf - Hadoop configuration object

Returns:

A compression codec object

getPreferredCompressionCodec

public static org.apache.hadoop.io.compress.CompressionCodec getPreferredCompressionCodec(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Returns the preferred compression codec. See {getPreferredCompressionCodec(Configuration)}.

Parameters:

context - Hadoop context object

Returns:

A compression codec object