datafu.pig.sessions
Class Sessionize

java.lang.Object
  extended by org.apache.pig.EvalFunc<org.apache.pig.data.DataBag>
      extended by datafu.pig.sessions.Sessionize
All Implemented Interfaces:
org.apache.pig.Accumulator<org.apache.pig.data.DataBag>

public class Sessionize
extends org.apache.pig.EvalFunc<org.apache.pig.data.DataBag>
implements org.apache.pig.Accumulator<org.apache.pig.data.DataBag>

Sessionizes an input stream. This UDF takes a constructor argument which is the session timeout (an idle period of this amount indicates that a new session has started) and assumes the first element of the input bag is an ISO8601 timestamp. The input bag must be sorted by this timestamp. It returns the input bag with a new field, session_id, that is a GUID indicating the session of the request. Example:

 %declare TIME_WINDOW  30m
 
 define Sessionize datafu.pig.sessions.Sessionize('$TIME_WINDOW');

 -- sessionize the visit stream
 VIEWS = group VIEWS by member_id;
 SESSIONS = foreach VIEWS {
   VISITS = order VIEWS by visit_date;
   generate FLATTEN(Sessionize(VISITS)) as (visit_date,member_id,url,session_id); 
 }

 -- count the number of sessions hitting the url
 ROLLUP = group SESSIONS by url;
 RESULT = foreach ROLLUP generate group as url, COUNT(SESSIONS) as session_cnt;
 
 


Field Summary
 
Fields inherited from class org.apache.pig.EvalFunc
log, pigLogger, reporter, returnType
 
Constructor Summary
Sessionize(java.lang.String timeSpec)
           
 
Method Summary
 void accumulate(org.apache.pig.data.Tuple input)
           
 void cleanup()
           
 org.apache.pig.data.DataBag exec(org.apache.pig.data.Tuple input)
           
 org.apache.pig.data.DataBag getValue()
           
 org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)
           
 
Methods inherited from class org.apache.pig.EvalFunc
finish, getArgToFuncMapping, getCacheFiles, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, isAsynchronous, progress, setPigLogger, setReporter, warn
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Sessionize

public Sessionize(java.lang.String timeSpec)
Method Detail

exec

public org.apache.pig.data.DataBag exec(org.apache.pig.data.Tuple input)
                                 throws java.io.IOException
Specified by:
exec in class org.apache.pig.EvalFunc<org.apache.pig.data.DataBag>
Throws:
java.io.IOException

accumulate

public void accumulate(org.apache.pig.data.Tuple input)
                throws java.io.IOException
Specified by:
accumulate in interface org.apache.pig.Accumulator<org.apache.pig.data.DataBag>
Throws:
java.io.IOException

getValue

public org.apache.pig.data.DataBag getValue()
Specified by:
getValue in interface org.apache.pig.Accumulator<org.apache.pig.data.DataBag>

cleanup

public void cleanup()
Specified by:
cleanup in interface org.apache.pig.Accumulator<org.apache.pig.data.DataBag>

outputSchema

public org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)
Overrides:
outputSchema in class org.apache.pig.EvalFunc<org.apache.pig.data.DataBag>


Matthew Hayes, Sam Shah