How To Populate A Postgresql Database With Mrjob And Hadoop

I would like to populate a database of Postgresql by using a mapper with MrJob and Hadoop 2.7.1. I currently using the following code: # -*- coding: utf-8 -*- #Script for storing t

Solution 1:

Each mapper needs its own database connection. Create the database connection in mapper_init() and close it in mapper_final(). You need to create the database separately from the mrjob script. You should try some very simple mrjob scripts first. You haven't started it the correct way. Work through the examples in the documentation.