MySQL Basic

Download MySQL Slide : Link




Transfer DB from MySQL to Hive by Sqoop :

—sqoop import --connect "jdbc:mysql://127.0.0.1:3306/DBNAME" --table TABLENAME --username USERNAME --password PASSWORD  --direct --hive-import --hive-database DSTABASE --hive-table TABLENAME -m 1

Hadoop WebHDFS REST API

—List directory

—curl -i "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/?op=LISTSTATUS"

—List the status of a file

—curl -i "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/temp.txt?op=GETFILESTATUS"

—Read a file

—curl -i -L "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/temp.txt?op=OPEN"

—Make new directory

—curl -i -X PUT "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/tmp?op=MKDIRS&permission=711“

—

—Rename  file

—curl -i -X PUT "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/temp.txt?op=RENAME&destination=/BDSClass/temp2.txt"

—Write a file

—curl -i -X PUT -L "http://$<Host_Name>:$<Port>/webhdfs/v1/BDSClass/tmp/UpTemp.txt?op=CREATE" -T temp.txt

Example Code :

• Apache HttpClient project : —http://hc.apache.org/httpclient-3.x/

public class JavaREST_01 {

  public static void main(String[] args) throws ClientProtocolException, IOException {

  CloseableHttpClient client = HttpClients.createDefault();

  HttpGet request = new HttpGet( URL);

  HttpResponse response = client.execute(request);

  BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));

  String line = "";

  while ((line = rd.readLine()) != null) {

            System.out.println(line);

  }

    }

}