Difference between revisions of "GPU621/DPS921 Could 4U"

From CDOT Wiki
Jump to: navigation, search
(Start a new project)
(Cloud 4U)
Line 28: Line 28:
  
 
Reduce: com.google.appengine.tools.mapreduce.Reducer; com.google.appengine.tools.mapreduce.ReducerInput;
 
Reduce: com.google.appengine.tools.mapreduce.Reducer; com.google.appengine.tools.mapreduce.ReducerInput;
 +
 +
== Program Profile ==
 +
 +
Nothing about parallelism here. This is an interface that communicates different parts of program and join them together. There are 3 sections here: mapping pattern, reduce pattern and file port (where the data came from). Platform Library will be pre-loaded in ever instance and no need to call. Default user interface calling section is included in every project.
 +
 +
[[File:profile.png|600px]]
  
 
== Map and Reduce Function ==
 
== Map and Reduce Function ==
 +
Map function:
 +
 +
[[File:map.png|600px]]
 +
 +
Reduce function:
 +
 +
[[File:reduce.png|600px]]
 +
 +
== Conclusion ==

Revision as of 13:38, 8 December 2016

Cloud 4U

Group Members

  1. Hualiang Xu, Everything.

Progress

What is Google App Engine

Google App Engine is a platform for building scalable web applications and mobile backends. It has a efficient Map Reduce Library that processes large set of data in map and reduce pattern in parallelism. This project will use this library to process a serious of searching history and produce a recommendation list for user.

Map and Reduce Library on GAE

In Map and Reduce Library, user code is only required for mapping and reduce function. The platform will shuffle and rearrange data order to make the upcoming reduce process extremely fast. This also reduces a lot of development work. For further boosting, the shuffle part can be rewritten to boost up the speed.

The original source code is here.

MPL.png

Start a new project

This link is a completed setup guide by Google.

Basic GAP project with Map and Reduce Library.

Strc.png

In this project, one more step is to import Map and Reduce Library in java files.

Map: com.google.appengine.tools.mapreduce.Mapper;

Reduce: com.google.appengine.tools.mapreduce.Reducer; com.google.appengine.tools.mapreduce.ReducerInput;

Program Profile

Nothing about parallelism here. This is an interface that communicates different parts of program and join them together. There are 3 sections here: mapping pattern, reduce pattern and file port (where the data came from). Platform Library will be pre-loaded in ever instance and no need to call. Default user interface calling section is included in every project.

Profile.png

Map and Reduce Function

Map function:

Map.png

Reduce function:

Reduce.png

Conclusion