Python Developer - Web Scraping


<p>We are looking for software engineers to join our Delivery team to work on web crawler development with Scrapy, our flagship open source project.</p><br /><p>Are you interested in building web crawlers harnessing the Scrapinghub platform, which powers crawls of over 3 billion pages a month?</p><br /><p>Do you like working in a company with a strong open source foundation?</p><br /><p><strong>Job Responsibilities:</strong></p><br /><ul><br /><ul><br /><li>Design, develop and maintain Scrapy web crawlers</li><br /><li>Leverage the Scrapinghub platform and our open source projects to perform distributed information extraction, retrieval and data processing</li><br /><li>Identify and resolve performance and scalability issues with distributed crawling at scale</li><br /><li>Help identify, debug and fix problems with open source projects, including Scrapy</li><br /></ul><br /></ul><br /><p>Scrapinghub&rsquo;s platform and Data offerings have been growing tremendously over the past couple of years but there are a lot of big projects waiting in the pipeline, and in this role you would be a key part of that process.</p><br /><p>Here&rsquo;s what we&rsquo;re looking for:</p><br /><br /><br /><br /><p><strong>Requirements</strong></p><br /><br /><ul><br /><li>2+ years of software development experience.</li><br /><li>Solid Python knowledge.</li><br /><li>Familiarity with Linux/UNIX, HTTP, HTML, Javascript and Networking.</li><br /><li>Good communication in written &amp; spoken English.</li><br /><li>Availability to work full time.</li><br /></ul><br /><p><strong>Bonus points for:</strong></p><br /><ul><br /><li>Scrapy experience is a big plus.</li><br /><li>Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, NLTK, pandas, scikit-learn, mapreduce, nosql, etc).</li><br /></ul>


Scraping the Web with open source!
🌃 87 cities
🌎 27 countries
⌚️ 31 timezones



Remote work at Scrapinghub