Feeds:
Posts
Comments

Archive for November, 2017

I’m inspired by Sreekar’s blog here to write this blog post. This article will help you create a VM instance in Google Cloud and install Datastax Enterprise software for learning purposes.

Below are the simple steps involved in this process.

  1. Sign up for Google Cloud for free. You will get $300 worth of bonus rewards to spend on Google cloud services.
  2. Screenshots for setting up Google cloud are in Sreekar’s post.
  3. Find “Compute Engine” dashboard link inside Compute section and click on it.
  4. Click on Create Instance.
  5. Select a VM instance with 3 cores, 15 GB RAM and CentOS 7 and create the instance.
  6. Note down your internal and external IP addresses.
  7. Click on the new instance link that just got created in the previous step.
  8. Click on drop-down link next to SSH under Remote access and select open in new browser window option.
  9. It opens up a terminal window inside the browser.
  10. Run below commands to install java.

sudo yum install java-1.8.0-openjdk.x86_64
sudo yum install net-tools

11. Create an account in https://academy.datastax.com/ if you haven’t already had one        12. Decide which DSE version you are going install. I have picked 4.8.13 for this example. You can find additional details on versions and installation instructions in Datastax website. I have included necessary installation instructions on this blog.

https://docs.datastax.com/en/datastax_enterprise/4.8/datastax_enterprise/install/installSilent.html

13. Use below command to download Datastax installer.

curl –user youremaildatastaxsite:yourpassword -O https://downloads.datastax.com/enterprise/DataStaxEnterprise-4.8.13-linux-x64-installer.run

chmod +x DataStaxEnterprise-4.8.15-linux-x64-installer.run

14. Create a file by name dse_install.props in the same folder where your installer is located. Place below contents inside the file.

vi dse_install.props

copy & paste

 

prefix=/usr/share/dse
enable-components=dse
install_type=simple
system_install=services_only
update_system=1
node_type=analytics
analytics_type=spark_integrated
cassandra_user=cassandra
cassandra_group=cassandra
start_services=1
ring_name=Test Cluster
listen_address=10.142.0.2
rpc_address=10.142.0.2

##Note: Replace above IP address with your internal IP address.

15. For additional installation properties, you can visit below Datastax link.

https://downloads.datastax.com/extra/sample_install_4.8.prop

16. Install DSE 4.8.13 by running below command

sudo ./DataStaxEnterprise-4.8.13-linux-x64-installer.run –optionfile dse_install.props –mode unattended

Cassandra is installed with cassandra as userID and password.

cd /usr/share/dse/bin

17. Below is the command stop Cassandra node.

sudo ./dse -u cassandra cassandra-stop

18. Below is the command to start Cassandra and Spark.

sudo ./dse -u cassandra cassandra -k

19. Below is the command run spark shell

sudo ./dse -u cassandra spark

20. Use below instructions to open firewall ports to see sparkmaster UI from you computer browser.

a) Search for “firewall rules” inside Google cloud console.

b) Click on create new firewall rule

c) Enter appropriate name and description.

d) Select ingress as direction for external traffic coming into the VM instance.

e) Enter 0.0.0.0/0 in source IP ranges.

f) Enter tcp:4000-5000; tcp:7080-7999 in specified protocols and ports.

21. You can access your spark master using

http://yourexternalipaddress:7080

 

There it is folks. You now have a working Datastax Cassandra + Spark that will allow you to play with Cassandra and Spark together.

In my next article, I will be posting simple examples to create sample table in Cassandra and play with Spark scripts.

 

Advertisements

Read Full Post »