At Cloudely, we work with a single mission: Transform the way clients experience Product & Implementation, Development, and Support.

Growth is a journey and never a destination. We are constantly thriving to grow in gaining the trust of clients globally in offering services across Salesforce, Oracle, Robotic Process Automation, DevOps, Web, and Mobile Programming to name a few. And we are just getting started!

We have fabulous opportunities for you to grow along with us! 

At Cloudely, you will get what you are looking for: the scope to learn, prove and grow. We are now actively seeking success-hungry candidates who want to grow in the domain of Data Lake.

Role: Data Lake Systems Engineer

Location: Canada

Summary:           
 Design, develop, and deliver solutions based on Big Data applications that fulfill the strategic vision for
enterprise applications to successfully support the business.  Activities will include:
 
 Perform the full deployment lifecycle for Big Data solutions, including installation,
configuration, initial production deployment, recovery, security and data governance for Data
Lake.
 Evaluates and provides technical solutions to design, develop, and support as required in a
lead role to business units wishing to implement an information technology solution. 
 Refine raw data into actionable insights in collaboration with Business and Data Scientist
using visualization and statistics with innovative analytics applications and systems.
 Develop applications that can interact with the data in the most appropriate way, from batch to
interactive SQL or low latency access using latest tools – Azure HDInsights, Azure Data
Factory, Azure Data bricks preferred.

 Essential Functions:
1. Leads implementation (installation and Configuration) of Data Lake with complete cluster
deployment layout with replication factors, setup access ADLS data, resource managers, node
managers & various phases of Workflows.  Experience with configuring workflows and
deployment using tools such as ADF is necessary.
2. Participates in design, development, validation, and maintenance of the Big Data platform and
associated applications.  Provides assistance in architecture oversight to how the platform is built
to ensure that it supports high volume / high velocity data streams and is scalable to meet growth
expectations.
3. Monitor workflows and job execution using the Azure Monitor , Ganglia or any equivalent tools. 
Assisting administration in commission and decommission of nodes, back up and recover data
using snapshots & high availability.
4. Develops, implements, and participates in designing column family schemas of Hive and HBase
within ADLS . Experience in designing flat and Star models with CBO impact analysis is
necessary.
5. Develops Data layers for performance critical reporting system.  Experience with real time big
data reporting system is necessary.
6. Recommends and assists with the development and design of ADLS –data partitioning,
Vectorization and bucketing with applicable query tools. Perform Day to Day operational tasks to
support custom functions or steps is required.
7. Develops guidelines and plans for Performance tuning of a Big Data /NoSQL environment with
underlying impact analysis of Spark jobs using CBO and analytical conversions. Implement a
mixed batch / near-real time architecture to analyze, index, and publish data for applications.
8. Develops efficient Spark code with various analytical functions. Experience with  SQL, advanced
Hive features like windowing, CBO,  views and ORC files and compression techniques are
necessary. Perform development of jobs to capture CDC (Change Data Capture) from Hive
based external and managed systems.
9. Partners with key internal teams, such as clinical and corporate operations and data
management, to ensure that the Big Data solution is identifying all the data points in upstream
systems and classifying them appropriately to support analytic objectives.  Identifies and
implements appropriate information delivery mechanisms that improve decision-making capability
of our customers.

10. Design, Develop and troubleshoot transformations to ingest and manipulate data from various
sources within the company and their extended environment using Open source technologies.   

  
Other Responsibilities:
 Designing and setting up exception handling jobs, writing Python, Scala, Bash scripts,
complex SQL queries, hierarchical, parent-child queries to support application systems.
 Providing Solutions for Portal and mash-up integration seamlessly connecting business
analytics with other applications in a publisher/subscriber model is a plus.

 
Qualifications
 
Requirements:
 
 BA/BS in computer science or similar discipline, plus 5+ years development experience in
technologies such as Spark, Azure  and Oracle databases .
 A very strong SQL/data analysis or data mining background , experience with Business
Intelligence, Data Warehousing, Solid understanding of large scale data management
environments (relational and/or NoSQL), Audit Controls, ETL Framework is expected.
 Prior experience in building scalable distributed data processing solutions with ADLS using
tools such as Data bricks ,  HBase (NoSQL), Hive, Spark, SOLR/Azure Cognitive Search,
Azure SQL, ADF
 Some proficiency with Map Reduce / Azure architecture, and Linux or Unix OS system
management and at least one scripting language experience is required. 
 Azure certified developers strongly preferred, but Strong spark Development is acceptable.
 Clinical or Financial Domain experience preferred.

The way to your dream job and organization is just a click away. Share your resume at [email protected]. To know more about us, please visit www.cloudely.com.