Guided Refactoring

Teach a Man To Fish:A Guided Refactoring of a Single ETL Data Source; END-TO-END

The best way to feel the power and difference between traditional and Big Data approaches is to follow an uninterrupted refactoring of a single use-case; from data landing, through transformation and into archive and reporting phases.



  • The engagement is a structured documented approach

  • Produces a detailed architecture of the approach illustrating the specific use-case and its configurations

  • Data workflow and phase breakdown/life-cycle

  • Monitoring and job recovery elaboration/configuration

  • Documentation and configuration of all processing layer dependencies by phase breakdown/life-cycle



The client receives a guided and documented “Best Practice” approach, implementation and training covering a single source of data, through its life-cycle on Hadoop and into the reporting layer.  The client will gain the knowledge of HOW and WHY the approach chooses the layers and methods that it does to get the job done. 



  • Duration:  4 – 11 weeks

  • Staffing:  .5 Sr. Architect, .5 Business Analyst, 1 Sr. Hadoop Data Process Engineer, 1 Sr. BI Engineer

  • Cost:  $xx.xx - $xx.xx

  • Delivery Input:

    • Design/Collaboration meetings with Business, IT Staff, Data Source Staff

    • Identified Data Source, Final BI Data Use Description

    • Functioning Hadoop cluster with dependent tooling layers

  • Delivery Artifacts:

    • Guided meetings with client and vendors

    • Solution Architecture Documents:

      • Approach Description and Goal Alignment Statement

      • Sequence Diagrams/Descriptions

      • Layer Collaboration Diagrams/Descriptions

      • Layer elaboration/configuration Diagrams/Descriptions

      • Job Flow and Recovery Design

      • Example analytic or relational data artifact generation and use from specific source data flow

    • Functioning “in-production” end-to-end implementation of best practice for a single data source

    • Delivery artifact education and meeting at hand-off points