HQ : 80-83 Long Lane, EC1A 9ET, London,UK

UK (+44) 020 7692 0834 | US +1-877-283-2691 contact@silicontern.com

Month: February 2017

Hortonworks : How to downgrade Ambari ?

Hortonworks : How to downgrade Ambari ? If you want to downgrade ambari and you are in prod env, here are some steps that can help you on doing that 1- first make sure under /etc/yum/yum.repos.d/ you have 2 files for ambari , one pointing to the new one , […]

Read More

Post kerberos install config – views – hive shell – beeline – hbase ..

Post kerberos install config – views – hive shell – beeline – hbase .. We need need to add keytab to .bashrc and bach_profile for hdfs , hive etc each connection will execute the command – Silent connection kinit -kt hive.service.keytab $(hostname) -f # uniquement la ou hive est installe kinit -kt hdfs.service.keytab […]

Read More

Hortonworks , Spark error : v1.6.2

Hortonworks , Spark error : v1.6.2 WARN TaskSetManager: Lost task 5.0 in stage 0.0 (TID 2, XXX.mc2.XXX.fr): java.io.InvalidClassException: org.apache.spark.rdd.RDD; local class incompatible: stream classdesc serialVersionUID = 7185378471520864965, local class serialVersionUID = 5009924811397974881 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:621) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) […]

Read More

Checks Before upgrade to HDP 2.5.3

Checks Before upgrade to HDP 2.5.3 Most of developers i have seen uses Maven central or Nexus jars ( because they can access internet from their local machines when they build jars through eclipse IDE ) , in this case they will face many issues after the upgrade – it’s […]

Read More

Hortonworks : how to add a datanode ?

First checks service iptables status service iptables stop chkconfig iptables off chkconfig | grep iptables Modify the /etc/hosts of the datanode vi /etc/hosts #add the ambari server FQDN IP FDQN SHORTNAME Modify the /etc/hosts of the ambari server vi /etc/hosts #add the ambari server FQDN IP FDQN SHORTNAME Check python #2.6.6++ […]

Read More

Kerberos HDP against IPA – SERVER using the ambari wizard

Kerberos HDP against IPA – SERVER using the ambari wizard First roll back any current kerberos setup on your cluster then visit http://XXXX:8080/#/experimental and enable the freeipa kerberos wizard make sure you  machines are resolvable by the configured DNS server under /etc/resolve.conf Now on the IPA server UI at https://XXXXXXX/ipa/ui/ […]

Read More

Python 3.5 setup in Prod zone

Python 3.5 setup in Prod zone The production zone has no access to download material from the public internet. To upload files to machines in prod we must use the dedicated FTP server. Here are the steps I took to configure a python 3.5 installation on VM HOST in the […]

Read More

HDP2.5 repo setup in Prod zone

HDP2.5 repo setup in Prod zone download the HDP2.5 tarballs to a dedicated store on the VM ; i.e use  /hadoop/store. this directory will need roughly 10 gb volume 1 connect to FTP server 2 ftp get HDP2.5 , AMBARI and HDP-UTILS tarballs 4 exit  # exit the ftp server […]

Read More

Tutorial : How to connect to hive server via beeline & knox gateway ?

Tutorial : How to connect to hive server via beeline & knox gateway ? In this instance sine I did not have a company server cert to install to knox gateway.jks i created a local truststore to be used by the client [root@host ~]# cd /var/lib/knox/data-2.5.3.0-37/security/keystores/ [root@host keystores]# /usr/jdk64/jdk1.8.0_112/jre/bin/keytool -export […]

Read More

Hadoop commands : All in one page

( This page is continuously updated ) Oozie basic commands Here below you will find a list of oozie commands that you may need when troubleshooting Oozie. #List all jobs ( workflow of coordinator ) oozie jobs -oozie https://XXXXXXX:11443/oozie -jobtype=coordinator -len=100000 | grep -i RUNNING # Check logs for a job $>oozie […]

Read More
Connection failed: Access denied for user 'talendexpertdb'@'n3nlwpweb007.prod.ams3.secureserver.net' (using password: YES)