Hi ,
I have more than 4+ years of experience in hadoop technologies like HDFS, MapReduce, Spark, sqoop. Kafka, data lake, pig etc.
Contact me for more detail
• Sqooping from different databases like netteza and oracle.
• Cluster to cluster migrations as a part of reporting DB sync and schema mismatch testing.
• Test automation fitnesse tool for query automation.
• Written the Java scripts in Spark process the data.
• Used spark sql with hive integrated quries result to RDD’s, Dataframes,datasets.
• Oozie workflows with control DB setup and eventing through tico.
• Involved in Unit testing.
• Created Hive tables to store the processed results in a tabular format.
• Developed the sqoop scripts in order to make the interaction between Pig and MySQL Database.
• Involved with the planning team in gathering the requirements, designing, development and testing.
• Automated shell scripts for moving scripts to PROD.
• Worked with dataset with pipe delimited and csv files.
• Reading and write data to the tables in hive with datacleansing operation.
• Converting impala oozie to spark jobs improving performance to as great extent.
Hi, my name is vinay. i am happy to help form your project. let me share my expertise on hadoop ecp systems.
basically i am a working as Big Data Developer for past 2 years.
Capable of transforming and processing large sets of structured, semi-structured into refined data based on business requirements. As per your requirements i'm expert in spark,scala and hadoop eco systems but i don't have any idea on SIEM parse.
clear explanation is mandatory.
Direct message for price is negotiation.