Thursday, 14 June 2018

Redis Cache Server - Setup/Testing/Java Code

Download stable version .tar/zip from here :https://redis.io/download
Follow this link for setup/testing : https://redis.io/topics/quickstart

Some commands to setup:

cd redis-stable
make
sudo make install
cd src
redis-server

Test it:
$ redis-cli ping
PONG ==> response

$ redis-cli                                                                
redis 127.0.0.1:6379> ping
PONG
redis 127.0.0.1:6379> set mykey somevalue
OK
redis 127.0.0.1:6379> get mykey
"somevalue"

pom.xml

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>redis</groupId>
    <artifactId>redis</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <!-- https://mvnrepository.com/artifact/redis.clients/jedis -->        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind -->        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.9.5</version>
        </dependency>

    </dependencies>

</project>

Java for Testing:

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;

public class Redisclient {

    public static void main(String[] args) {

        //Connecting to Redis server on localhost        Jedis jedis = new Jedis("localhost");


        //Can read jedis objects from Pool.
        /*JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();        JedisPool pool = new JedisPool(jedisPoolConfig,                Constants.REDIS_IP_EUROPE,                Constants.REDIS_PORT,Constants.REDIS_TIMEOUT);        Jedis jedis = pool.getResource();        jedis.auth(Constants.REDIS_PWD);*/
        //Get all Student details from DB. Here we can use Tax calculation values.        List<Student> list = getAllStudentsFromDB();

        ObjectMapper mapper = new ObjectMapper();
        String studentJSON = "";
        try {
            studentJSON = mapper.writeValueAsString(list);
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }

        System.out.println("JSON : "+studentJSON);

        //writing value to redis        jedis.set("allstudents", studentJSON);

        //reading from redis        String allstudents = jedis.get("allstudents");

        try {
            List<Student> studentList = mapper.readValue(allstudents,new TypeReference<List<Student>>(){});
            studentList.forEach(System.out::println);

        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static List<Student> getAllStudentsFromDB() {

        Student st =  new Student();
        st.setAge(10);
        st.setId(1);
        st.setName("kumar");
        List<String> subjects = new ArrayList<String>();
        subjects.add("maths"); subjects.add("social"); subjects.add("science");
        st.setSubjects(subjects);

        Student st1 =  new Student();
        st1.setAge(110);
        st1.setId(11);
        st1.setName("kumar1");
        List<String> subjects1 = new ArrayList<String>();
        subjects1.add("maths1"); subjects1.add("social1"); subjects1.add("science1");
        st1.setSubjects(subjects1);

        List<Student> list = new ArrayList<Student>();
        list.add(st); list.add(st1);

        return list;
    }
}

Schedule Jobs/Cron Jobs - Linux machines

1 crontab


The command to create/edit, list, and remove cron jobs is crontab. If you call it with the -u option, it specifies the name of the user whose crontab is to be tweaked. If this option is not given, crontab examines "your" crontab, i.e., the crontab of the person executing the command. If you are looged in as root and run crontab without -u, then root's crontab is listed/modified/removed. If you are logged in as exampleuser and run crontab without -u, then exampleuser's crontab is listed/modified/removed.

Examples:
crontab -l
lists the cron jobs of the user as that you are currently logged in:
server1:~# crontab -l
* * * * * /usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log
30 00 * * * /usr/local/ispconfig/server/cron_daily.sh > /dev/null 2>> /var/log/ispconfig/cron.log
server1:~#
crontab -u exampleuser -l
lists all cron jobs of exampleuser.
crontab -e
let's you create/modify the cron jobs of the user as that you are currently logged in (I'll come to the syntax in the next chapter).
crontab -u exampleuser -e
let's you create/modify the cron jobs of exampleuser.
crontab -r
deletes all cron jobs of the user as that you're currently logged in.
crontab -u exampleuser -r
deletes all cron jobs of exampleuser.
If you have written your cron jobs to a text file, you can use the text file to create the cron jobs. For example, let's assume you have created the text file /tmp/my_cron_jobs.txt...
vi /tmp/my_cron_jobs.txt
... with the following contents:
30 00 * * * /path/to/script
You can create a cron job from that file as follows:
crontab /tmp/my_cron_jobs.txt
(Or for exampleuser:
crontab -u exampleuser /tmp/my_cron_jobs.txt
)
Please note that this will overwrite all previously created cron jobs - if you've already created some cron jobs, you better use crontab -e and add the new cron job manually.
See
man crontab
to learn more about the crontab command.

2 Cron Job Syntax

A cron job consists out of six fields:
<minute> <hour> <day of month> <month> <day of week> <command>
              field          allowed values
              -----          --------------
              minute         0-59
              hour           0-23
              day of month   1-31
              month          1-12 (or names, see below)
              day of week    0-7 (0 or 7 is Sun, or use names)
When specifying day of week, both day 0 and day 7 will be considered Sunday.
A field may be an asterisk (*), which always stands for first-last.
Names can also be used for the "month" and "day of week" fields. Use the first three letters of the particular day or month (case doesn't matter), e.g. sun or SUN for Sunday or mar / MAR for March..
Let's take a look at the two cron jobs from the first chapter:
* * * * * /usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log
This means: execute /usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log once per minute.
30 00 * * * /usr/local/ispconfig/server/cron_daily.sh > /dev/null 2>> /var/log/ispconfig/cron.log
This means: execute /usr/local/ispconfig/server/cron_daily.sh > /dev/null 2>> /var/log/ispconfig/cron.log once per day at 00:30h.
The day of a command's execution can be specified by two fields: day of month, and day of week. If both fields are restricted (i.e., aren't *), the command will be run when either field matches the current time. For example, 30 4 1,15 * 5 would cause a command to be run at 4:30h on the 1st and 15th of each month, plus every Friday.
You can use ranges to define cron jobs:
Examples:
1,2,5,9 - means every first, second, fifth, and ninth (minute, hour, month, ...).
0-4,8-12 - means all (minutes, hours, months,...) from 0 to 4 and from 8 to 12.
*/5 - means every fifth (minute, hour, month, ...).
1-9/2 is the same as 1,3,5,7,9.
Ranges or lists of names are not allowed (if you are using names instead of numbers for months and days - e.g., Mon-Wed is not valid).
1,7,25,47 */2 * * * command
means: run command every second hour in the first, seventh, 25th, and 47th minute.
Instead of the first five fields, one of eight special strings may appear:
              string         meaning
              ------         -------
              @reboot        Run once, at startup.
              @yearly        Run once a year, "0 0 1 1 *".
              @annually      (same as @yearly)
              @monthly       Run once a month, "0 0 1 * *".
              @weekly        Run once a week, "0 0 * * 0".
              @daily         Run once a day, "0 0 * * *".
              @midnight      (same as @daily)
              @hourly        Run once an hour, "0 * * * *".
You can also use name=value pairs in a crontab to define variables for the cron jobs:
# use /bin/bash to run commands, instead of the default /bin/sh
SHELL=/bin/bash
# mail any output to exampleuser, no matter whose crontab this is
MAILTO=exampleuser
# set the PATH variable to make sure all commands in the crontab are found
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

* * * * * my_command
Please note: unless you set a PATH variable in a crontab, always use full paths in the crontab to make sure commands are found and can be executed. For example, instead of writing rsync, you should write /usr/bin/rsync. Use which to find out the full path of a program:
which rsync
server1:~# which rsync
/usr/bin/rsync
server1:~#
See
man 5 crontab
to learn more about the cron job syntax.

Load Balancer / Cluster - Algorithms for balancing

A computer cluster is a single logical unit consisting of multiple computers that are linked through a LAN. The networked computers essentially act as a single, much more powerful machine. A computer cluster provides much faster processing speed, larger storage capacity, better data integrity, superior reliability and wider availability of resources.

Computer clusters are, however, much more costly to implement and maintain. This results in much higher running overhead compared to a single computer.

Many organizations use computer clusters to maximize processing time, increase database storage and implement faster data storing & retrieving techniques.
There are many types of computer clusters, including:
  • Load-balancing clusters
  • High availability (HA) clusters
  • High performance (HP) clusters
The major advantages of using computer clusters are clear when an organization requires large scale processing. When used this way, computer clusters offer:

  • Cost efficiency: The cluster technique is cost effective for the amount of power and processing speed being produced. It is more efficient and much cheaper compared to other solutions like setting up mainframe computers.
  • Processing speed: Multiple high-speed computers work together to provided unified processing, and thus faster processing overall.
  • Improved network infrastructure: Different LAN topologies are implemented to form a computer cluster. These networks create a highly efficient and effective infrastructure that prevents bottlenecks.
  • Flexibility: Unlike mainframe computers, computer clusters can be upgraded to enhance the existing specifications or add extra components to the system.
  • High availability of resources: If any single component fails in a computer cluster, the other machines continue to provide uninterrupted processing. This redundancy is lacking in mainframe systems.

Algorithms for balancing is one of the most important factors in this context, then we will explain three basic methods :
Least Connections
This technique redirects the requests to the lowest based on the number of requests / server connections. For example, if server 1 is currently handling 50 requests / connections, and server 2 controls 25 requests / connections, the next request / connection will be automatically directed to the second server, since the server currently has fewer requests / connections active.
Round Robin
This method uses the technique of always direct requests to the next available server in a circular fashion. For example, incoming connections are directed to the server 1, server 2 and then finally server 3 and then the server 1 returns.
Weighted Fair
This technique directs the requests to the load based on the requests of each and the responsiveness of the same (performance) For example, if the servers server 1 is four times faster in servicing requests from the server 2, the administrator places a greater burden of work for the server 1 to server 2.
Refer:
https://www.esds.co.in/blog/cluster-computing-definition-and-architecture-of-a-cluster/#sthash.SEcCghTH.dpbs