DESIGNING THE LAYER This step involves conceptualizing the solution and defining the components required for implementation Storage: Bring data to an internal storage layer. This is a very important step, considering that the sources are purely for data transfer and there is no guarantee of persistence or availability. Identification: We receive multiple types of data from our partners. Every type of data has its own input definition, output definition and processing steps. There was a need to identify the type of data and triage it to the corresponding data processor. Normalisation: Process the files as soon as the data is sent by the partners. The data is normalised to match the schema required by downstream components. The normalised files would then be picked by scheduled batch processing jobs DATA INGESTION COMPONENTS File Watcher: Identifies new files and triggers data processing jobs Workflow Orchestrator: Runs defined data processing workflow to [...]

Many of us are familiar with Maslow’s Hierarchy of Need – only second to the physiological needs of food and water, is the need for Security and Safety. If the last few weeks have taught us anything, it’s that uncertainty is the new normal. However, I still choose to look at the silver lining, this too shall pass and we will wake up to a new world when this is over. […]

Your resume is the first thing that a recruiter sees to start making an impression about you. It is important to follow a few basic steps to get that coveted job. In a market flush with talent, make sure you stand out! Brevity Brevity on a resume should be exercised in two ways. One, keep your resume to one page only – even highly successful professionals keep their resume to a page. Second, don’t use long sentences or paragraphs to describe anything – try to keep every bullet point to one line. Email address Coolguy93@abcd.com (as an example) is a lot less likely to get a response than your firstname@abcd.com Tailor your resume for the opportunity Obviously, possessing the right skills and experience is paramount, but highlighting the right industry keywords and customizing the resume to match the job description is equally important to get noticed. Relevant experience & educational qualifications [...]