Junior Data Engineer 初级爬虫工程师

Job description

岗位职责

1. 收集和处理原始数据(包括编写脚本,Web抓取, API抓取, SQL查询);

2. 将非结构化数据处理为适合分析的形式;

3. 监控性能并建议任何必要的基础架构更改;

4. 测试数据,以保证机器学习模型中使用的数据转换和数据验证的准确性;

5. 为数据挖掘,数据建模和数据生产开发一套流程;

6. 与队友紧密合作,确保一致性并最大程度地利用数据。

  

Role Description

1. Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, writing SQL queries, etc.).

2. Process unstructured data into a form suitable for analysis.

3. Monitoring performance and advising any necessary infrastructure changes.

4. Testing and validation in order to support the accuracy of data transformations and data verification used in machine learning models

5. Develop set processes for data mining.

6. Collaborate closely with teammates to ensure consistency and maximize the use of data

 

Next to the jobs described above, we offer the opportunity to develop your area of interest and we’ll assign work to support your personal development within that area.

Job requirements

岗位要求

1. 计算机科学,工程或相关学科本科及以上学历;

2. 熟悉Python;具有SQL和NoSQL技术的经验优先;

3. 能够使用Scrapy/puppeteer开发网络爬虫优先;

4. 熟悉基本的反爬虫原理优先;

5. 英文熟练者优先;

6. 具有在软件/ ETL开发环境中工作的经验;

7. 对版本控制工具(例如git)有扎实的应用经历;

8. 愿意学习和在短时间内掌握新的技能和方法;擅长团队合作。

 

我们的优势:

- 在一支由65多人组成​​的国际化的年轻的工作团队;

- 我们曾被Google,麦肯锡和Rocket Internet授予欧洲最佳B2B创业公司;

- 轻松自由的工作氛围,互帮互助的工作团队和不加班的弹性工作模式;

- 各种有趣的团队活动,吃不完的零食喝不完的饮料;

- 办公地址位于上海市中心办公,生活便利。

 

Job requirements

1. Bachelor's Degree or above in Computer Science, Engineering or related subjects;

2. Familiar with Python; Experienced with SQL and NoSQL technologies is preferred;

3. Experienced with building web spiders with scrapy/puppeteer, is preferred;

4. Familiarity with commonly used anti-crawling methods is preferred;

5. Solid understanding of version control tools (e.g. git);

6. Proficient in English is highly preferred

7. Fast learner and open-minded to new technologies and methodologies and team-oriented.

 

What’s in it for you

- Great office location right in the center of Shanghai

- Working within an international team of over 65 people that truly values your contribution

- Growing company full of opportunities & awarded by Google, McKinsey, and Rocket Internet for best B2B startup in Europe

- An awesome culture of responsibility and the freedom to turn your ambition into reality - regardless of your role and level

- Exciting work atmosphere with no shortage of snacks, drinks, birthday treats, and social events

- Monthly team events and weekly Friday company catch-ups and drinks

 

About us

Dashmote is a start-up focusing on the next generation of data products powered by AI technology, with offices in Amsterdam (HQ) and Shanghai. We connect the offline and online worlds by decoding the digital footprint of locations, allowing our enterprise clients to understand the market and make smarter decisions.

With about 60 employees, Dashmote has ambitious plans in the upcoming years and therefore we need to make sure that we have the right people in place to put such plans into practice. Do you want to boost your career by contributing to Dashmote’s core product, used by some of the biggest multinational companies? Then we're looking for you!