Data Engineering (AO):

Must Have:

• Python, Spark Or PySpark
• Hadoop
• Cloud  (AWS/Azure/GCP/Databricks)
  • Glue - ETL, Data Catalog
  • EMR
  • Redshift
  • S3
  • DBT
• Batch Processing
• SQL
• Experience with Streaming
• Integrations & Data Interoperability

Nice to have :

• Infra - K8s, Docker, ECS/EKS, DevOps/DataOps, Ansible, Terraform
• Java/Scala
• Streaming - Kafka, Flink
• CT (ability to test the pipeline, UT framework, Integration Testing

Pls find the screening questions below for Data Engineer:

For Recruiter:

1. Need to look for terms like informatica, Data storage, MicroStrategy, Talend- Should NOT be repeated frequently in jobs else its drop
2. Best Combo: Python – Hadoop – Spark -Cloud (Aws/GCP/Azure/Databricks).
3. Python is compared to SQL as backend.

From Candidates:

1. How conversant are you with Python/Scala on a scale 1-5
2. Would you be interested to take a data structures and Algorithm test in any programming language of your choice
3. What Development Techniques do you adopt for your data project?
4. Explain in brief what measures were taken to assure data quality.
5. What data Modelling approach did u adopt for enabling Business Intelligence If any
6. What technical gaps do you want to work on once you join our organisation?