Data Engineer – Web Scraping
Application deadline 3 months ago: Saturday 15 Oct 2022 at 23:59 UTCOpen application form
Result of Service The individual is expected to fulfill the following goals: - Maintaining quality applications. - Designing and launching new features. - Maintaining code Integrity. - Delivering meticulous and well documented code. - Implementing security and data protections schemes in all its developed back and front-end applications.
Work Location Beirut
Expected duration Expected duration of 6 months with possibility of extension depending on needs and funding availability.
Duties and Responsibilities Under the overall guidance of the Chief, ICTS, and direct supervision by the team leader, applications unit, the contractor will perform the following tasks: - Help to design and implement the data crawling architecture and a large-scale crawling system - Work on developing not just large-scale scraping tools and APIs but also data integrity, health, and monitoring systems. - Collaborate with our product and business teams to understand / anticipate requirements to strive for greater functionality and impact in our data gathering systems. - Design, implement, and maintain various components of our data infrastructure.
Qualifications/special skills Skills: A minimum of one year experience with Python for data wrangling and cleaning is required. Advanced experience with SQL (MS SQL Server preferred) is required. Proficiency in crawling & scraping using any libraries such as Scrapy, BeautifulSoup, and Selenium is required. Experience extracting data from multiple disparate sources including Web, PDF, and spreadsheets is required. Productionized experience with techniques and tools for crawling, extracting, and processing data is required. Sound Knowledge in bypassing Bot Detection Techniques is required. Experience using HTTP Proxy techniques to protect web scrapers against site ban, IP leak, browser crash, CAPTCHA and proxy failure is required. Academic Qualifications: A Bachelor's degree in Computer Science, Engineering or any other relevant field is required. Experience: A minimum of one year of qualifying work experience is required. Language: English and French are the working languages of the United Nations Secretariat. For this position, fluency in English is required. Knowledge of a second official United Nations language is an advantage
Additional Information ESCWA is looking for a Web Scraping focused Data Engineer; the individual will be responsible for extracting and ingesting data from websites using web crawling tools. In this role, the individual will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports, and data management.
No Fee THE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.