Wednesday, 27 August 2014

Computer Hacking - Things you need to know- How and Prevention

Computer Hacking - Things you need to know- How and Prevention

This blog is about the general network/data security knowledge that everyone working in an office setup with desktops and laptops should know. In an Internet age, with every computer hooked to internet, it has become very easy for any one to gain access and cause havoc. Some awareness of internet security can go a long way in preventing such security breaches.

Damages from Internet/Data security breach and computer hacking:

a) Data theft/Loss:  Lets says your credit card number is stolen, then someone can hack into bank account. If your computer password in stolen, someone can delete or steal data from your computer.

b) Loss of customer trust: If you are working for an organization or running an organization that stores customer data, then any theft or loss of customer data can damage the customer trust, bring law suits, and big financial loss.

c) Data/System availability: If someone hacks into your computing device then the hacker can bring the whole system down, install virus/backdoor programs and make the system unusable.

Common ways data security breeches and computer hacking occur:

a) Installing virus or Trojan programs to steal data by sending the virus programs through emails or making users download it automatically when they visit certain web sites.

b) Social engineering- Using the information available on internet or other sources, hackers can try to call the people impersonating as someone else to gather more information required to steal an organization or personal information.

c) Insider job- Majority of the data security breaches or intrusion happen with help of a person from within the organization. The person working in an organization has access to computing assets that can be used to steal data or whatever malicious purpose.

d) Hackers: These are folks who know to hack into IT networks and try various sophisticated ways to get into an organization data network. They can remotely monitor an organizations or personal network set up and find entry points to get into the network.

Prevention of Internet/Data security breach:

Personal users:
a) Keep all your computer passwords secure and strong. Don't give away your password to anyone. Always set up a password for your computer, phones, and tablets. Keep it strong by  making it hard to predict. Don't use your names as password. Use special characters such as #$% in your passwords and use mix of numbers, upper and lower case letters.

b) If you are working always lock your computer/phone before leaving your desk. Do not leave your phone on the desk. Do not write passwords on sheet of paper or yellow sticky and leave it on your desk.

c) Do not install programs from popups and websites that are fishy and not well known. Don't open email attachments that have executable programs. Always use anti virus and malware detection programs. Microsoft defender and security essential is a free anti virus program.

d) Apply the operating system security patches whenever available and prompted by computers to do so. Create a recovery CD of your operating system and if your computer is infected then do a fresh install of OS by reinstalling it from the recovery CD.

a) Every organizations these days have IT security polices and security devices to protect their organizational IT assets. Every employee should follow those security policies.

b) If you are a start up, then better to get the security software at the earliest and protect your network. Common security devices used by companies are Firewalls, Intrusion detections systems, security auditing programs, virus and malware scanners and detectors,etc. If you cannot affords costly devices, then use the open source programs available. 

c) Use VPN (virtual private networks) to connect to office network. VPNs encrypt data during transmission from outside to inside network. Use VPNs if you are a mobile user and connect frequently to office network. Always encrypt emails and any data sent outside the network.

d) Set proper folder permissions on your network so that users not belonging to your organization cannot access it.

e) If you are having any website, make it secure by hosting it on servers behind firewalls. Firewalls are security programs that blocks connection to certain ports of your server/network.

g) Security audit the computer network to identify programs that are more vulnerable to hacking, and find all the ways to secure your network.

f) Keep back up of data on data centers and have disaster recovery plans.

Saturday, 9 August 2014

How to migrate/deploy code from one environment to another in Informatica

There are mainly three ways to migrate/deploy code from one environment to another in Informatica. Those three ways are:

Method 1- Deployment groups- This involves creating a deployment group from the repository manager with all the informatica objects that need to be migrated to the destination machine. This most likey is used wherever there is version control used in the destination machine since the deployed objects can be rolled back to the previous version. The steps to create deployment group is below:

Go to Repository Manager.
Under Tools>Deployment Groups > Create
Provide a name for the deployment group and add the desired permissions.
Select the informatica objects that you want to migrate , right click and select add to deployment group or you can also copy the objects and paste it to the deployment group
Now copy this deployment group from source repository to the destination repository using normal copy and paste.
The deployment wizard will start and you have to follow the on screen steps

Method 2- Import/Export using XML- In this method all the objects (source/target definitions, mappings, workflows, etc) are exported as xml to local hard disk from the source repository using either workflow manager or repositor manager. The exported XMLs are imported into the destination machine. Make sure to import the source/target definitions before the mappings.

Method 3- Drap and drop/copy and paste method- This is the most simplistic one. Open the target and source repository in repository manager. You can do it with workflow manager too but repository manager is more better option. Copy first all the source/targets then you want to migrate and paste it to the destination machine. Next do the same with mappings, sessions, workflows in this order. The shortcuts need to be created in the designer.

Check more about migration issues in this article:

Informatica Migration/Implementation in production - common issues

I work in a medium sized company where everything is fast paced with little chance for error. In my first week at work, I had to implement some Informatica code. One of the guy who joined me got booted out because one of his implementation went a bit wrong. Can you sense the amount of pressure I had to go through? I feel it even after close to 2 years with that company.

Implementation of Informatica code or in general any code to production is very stressful. Having to work over the weekends, having to work with implementation folks, pressure to get issues fixed faster, etc all add to the stress that a developer has to go through. Every company has its own way of release management and deploying code. We have a system where the developer creates implementation document and the production support folks deploy it in production reading the instructions. If any error in the instructions, then pretty much the developer is screwed. If any changes have to be done, then we have to raise separate change tickets, get approved, etc. Very hellish. If the migration involves putting new code and not working with existing critical systems, then we get a bit of room to work around but if it involves critical systems then we have zero chance for any thing going wrong. So, I thought of writing this article to list common issues with implementing Informatica code to production. The assumption below is that you are not using deployment groups and doing a step by step migration by copying the objects to be migrated to the destination/production environment.

Version control- Thank god we do not have version control in our prod environment. We back up the informatica objects as xml before every migration. Version control makes migration even more difficult. If you migrate a mapping and if the source/target is not checked in then those source/targets wont make it to the destination machine. Make sure to check in all the objects that need to be migrated. Also, in the destination machine make sure those objects are checked in too.

Not creating proper source or target shortcuts in target environment-
If you copy a mapping that has source or target shortcuts, and if those shortcuts are not created prior to copying the mapping in the destination then the mapping gets copied with the shortcut removed. Whenever you are working with mappings that have shortcuts, please make sure to create those shortcuts in the target environment before migrating the mappings.

Impacting other mappings- It is always necessary to do a thorough dependency check on all the source/targets that are getting updated from a migration. The changes to source/target might make other mappings that use the same source/target to fail after the update is done. Also validate all the mappings that are impacted by the source/target definition changes.

Not validating impacted sessions- after a mapping change has been migrated make sure to refresh the session that was impacted. Otherwise the session will bomb when it runs in production.

Not having proper connections set up in production- If a matching database connection is not in production/target environment then most likely when you migrate the session it will show the non existence of the connection in the source environment. Most likely you have to create this connection in target environment or change the connection to match with a connection in the target environment. Also, some sessions are sensitive to the type of database connection.
Also proper permissions have to be set up for the connection objects. Otherwise it will fail with no permission to use the connection object. Have the right database drivers set up in the destination machines with all the correct tnsnames.ora or .odbc.ini entries.

Object Permissions- The folders in the destination should be set up with proper permissions so that the admin or the users of the destination machine can run the workflows and make changes if necessary.

Setting up the parameters/variables: Make sure to check if all the mapping/workflow variables were migrated properly. Also, make sure that the parameter file was migrated and accessible to the informatica workflows.

Scripts- Make sure all the unix/perl or whatever script you have going along with your informatica workflows are migrated, and Informatica can access and execute/read/write it.

Not having the database changes done correctly - If you migrate the mappings/sessions but the database changes are not done to reflect the new informatica code, then the sessions will fail. This could also include recreating some of the views of which the base tables where modified.

Copying reusable session that have overides in the workflows
--If the reusable session has overrides, we have observed that sometimes these overrides are gone when the reusable session is copied again from dev/non prod environment. So better check all the overrides such as data base connection, and other sessions properties are not changed after sessions are copied.

In my recent migration, I copied the mapping and just refreshed the session that was using this mapping in the destination. However, this mapping had a new lookup and the session did not have connection configured in the destination. I should have migrated the session too. Anyhow, had to set up this connection in the destination session. 

In another migration, I copied a reusable session to the destination and was hoping the non reusable session in the destination machine will automatically pick this reusable session. But it did not. Had to remove the session that previously existing and add the reusable session. Very naive error.

Scheduling conflicts- You will get every thing correct but then you realize that there is backup job running and your database is not available. WTF??? or some other informatica workflow is writing to the same table or using up all the database resources. Check the scheduling conflicts with other jobs.

Impact to downstream process
- Lets say you are generating a file for some downstream process to consume but made changes that the downstream process cannot process anymore then it will be a major issue. Check if all the downstream or upstream processes are working as expected.

I'm sure there are lot more that is not on top of my head. Will update this post whenever I remember.

Friday, 8 August 2014

Unix Interview Prep Tutorial- Part 1: Commonly used Unix Commands, usage and purpose:

Unix Interview Prep Tutorial- Part 1:Commonly used Unix Commands, usage and purpose:

Below are list of unix commands that you might need for daily use. This is more focused for beginners and intermediate unix users.The purpose of the command is mentioned in comments between /* and */.

PWD: command is used to show the present working directory.
Usage: pwd

ls: command is used to get list of all files in the directory
Usage: ls -l
       ls -ltr

wc: is used to get word count from a file.
Usage: wc inputfile.txt /* word count from a file */
       wc -l inputfile.txt /*line counts from a file */
       ls -l| wc -l /* to get number of files in a directory */

cd: commands is used to change the working directory.
Usage: cd pathtodirectory
       cd /usr/bin
       cd .. /* go one directory up */

mkdir: is used to create a directory:
Usage: mkdir <dirname>

chmod: is used to change the permissions on a directory. After you create a directory or file you might have to change the permission on the directory.
Usage: chmod 777 filename /* provide read/write/execute permission to all the users */
       chmod 555 filename /Previde read/execute permission to all the users */

More: command is used to shown the contents of a file.
more filename.txt

Head/ Tail: Head command is used to show the first few lines of a file and tail command is used to show the last few lines. You can also specify how many lines to be displayed;
Usage: head inputfilename.txt
       head -100 inputfilename.txt /* show first 100 lines of inputfilename.xt */
       tail inputfilename.txt
       tail -100 inputfilename.txt /* show last 100 lines of inputfilename.xt */

Grep: command is used search for a pattern string in a file:
grep "*2014" inputfile.txt  /*search for 2014 in your input file*/
grep "searchstring" inputfile.txt
grep 'scp' *  /*search scp in all the files in the current directory*/

gzip command is used to zip a file to .gz format or unzip a file.
gzip <filename> /* To create a .gz file*/
gzip -d <filename> /* to uncompress a .gz file

unzip command is used to create .zip files in unix.
Usage: unzip -l <> /* list all the files without uncompressing it */
       unzip <>   /* to unzip a file i.e to decompress a file to .zip format */
       unzip -c  <file.txt> /* to zip a file i.e to compress a file to .zip format */

touch command is used to create a file.
usage: touch filename.txt

rm command is used to remove a file. rmdir command is used to remove a directory.
usage: rm filename
       rmdir directoryname
       rm -f filename  /*Does not display error message. Good to use when removing files that might not exist through an automated script */.

sftp command is used to sftp to a sftp server
usage: sftp -o Port=22 username@serveraddress

date command is used to display the sytem date
usage: date
       date +'%Y-%m-%d %H:%M:%S'

echo command is used to print a text on the terminal
usage: echo "print something"
       echo "write to a file" > out.txt /* write the string to a file */

export command is used to export environment variables

sed command is used to replace a content of a file or extract contents of a file
sed "s/2014/2015/g" inputfile.txt > ./outputfile.txt /* replaces 2014 with 2015 in the inputfile.txt and writes it to outputfile.txt */
sed -n '3p' ./inputfile.txt /* read third line of the inputfile.txt file */

Cut command is used to extract sub string
Usage: cut -c 3-7 filename /* to output 3rd to 7th character of each line */
cut -d ":" -f 3- filename /* to extract 3rd fiedd field of each line using the : as the field delimiter */

diff command is used to compare two files:
Usage: diff file1 file2

du command is used to find about the disk space usage
Usag: du
du -sk

df command is used to find out the available disk space.
Usage: df
df -k

Also check: