Class

pegasus.spark.cc.utils

LongExternalSorter

Related Doc: package utils

Permalink

class LongExternalSorter extends AnyRef

Linear Supertypes

AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

LongExternalSorter
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new LongExternalSorter(basePaths: Array[String])

Type Members

class LocalPaths extends AnyRef

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
val BUFFER_SIZE: Int
val KEY_SHIFT: Int
def acquireNextBuffer(it: Iterator[Long]): Int

Fill the buffer with next long values.
Fill the buffer with next long values.
it
long iterator
returns
the number of values read
final def asInstanceOf[T0]: T0

Definition Classes
Any
val buffer: Array[Long]
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def getBufferIterator(limit: Int): Iterator[Long]

Iterator for the buffer.
Iterator for the buffer.
limit
to read
returns
long iterator from the buffer
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def getDirectReaderIterator(path: String): Iterator[Long]

Iterator for a sorted file.
Iterator for a sorted file. !IMPORTANT this operation delete the input file after it reads all values.
path
of a sorted file
returns
long iterator for the sorted file
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val localPath: LocalPaths
def localSort(it: Iterator[Long]): Queue[Iterator[Long]]

This operation reads long values from a given iterator, writes to the buffer, and sorts it.
This operation reads long values from a given iterator, writes to the buffer, and sorts it. If the input exceeds the size of the buffer, this operation spill the sorted buffer out and continues until it reads all long values.
it
a long iterator
returns
a series of sorted iterators which are parts of the input iterator
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def sort(it: Iterator[Long]): Iterator[Long]

Sort a given long iterator.
Sort a given long iterator. It consumes the given iterator and return a new one.
it
long iterator to be sorted
returns
sorted long iterator
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
var usedBufferSize: Int
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped