Informatica Java transformation to parse comma separated columns and generate new rows - normalizer

Informatica Java transformation to parse comma separated columns and generate new rows - normalizer limitations

Informatica normalizer transformation is normally used to convert a column containing multiple values into separate rows. i.e something like ID, 1,2,3,4 can be converted to something like below.
ID , 1
ID, 2
ID, 3
ID, 4

The problem with normalizer however is that it has to know the number of occurences. If the number of occurences are very high then it is hard to create the output ports for that many number of ports.

The easy way in these cases is to use Java transformation to parse the column containing multiple values and generate a separate row for each of them:

The java code is shown below:

String str_var= INPUT_PORT_NAME
String[] arr;
String delimiter = ",";
arr=str_var.split(delimiter);
for (int i=0; i<arr.length;i++) {
INPUT_PORT_NAME=arr[i];
generateRow();
}

Java code to extract tweets since any date using Twitter4J API

/* Below is the code to extract tweets since any date or any given date  using twitter 4J API */       


         Twitter twitter = new TwitterFactory().getInstance();
         twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_KEY_SECRET);
         String accessToken = 'asdasdsdsaasdsaedadasdadsadasd'; /* enter your twitter access token */
        String accessTokenSecret = '2020202-asdsadsadasdsadsadsadsa'; /enter your twitter secret token */
         AccessToken oathAccessToken = new AccessToken(accessToken, accessTokenSecret);
        twitter.setOAuthAccessToken(oathAccessToken);
        Query query = new Query("Iron Man");
        Date date = new Date();
        String modifiedDate= new SimpleDateFormat("yyyymmdd").format(date);
        query.setSince(modifiedDate);
        QueryResult result;
            do {
            result = twitter.search(query);
            List<Status> tweets = result.getTweets();
            for (Status tweet : tweets) {
                 System.out.println("@"+tweet.getUser().getScreenName() + "|" + tweet.getText()+"|"+ tweet.isRetweeted() );
                 }
              } while ((query = result.nextQuery()) != null);


To create twitter app account and get the access tokens, check out the link below:
http://dwbitechguru.blogspot.ca/2014/12/how-to-create-twitter-app-account-and.html

How to create twitter app account and create twitter applications that can download/upload tweets and then load to Hadoop?

How to create twitter app account and create twitter applications that can download/upload tweets and then load to hadoop?

To create any twitter application that can download or upload tweets to twitter, you first start with creating an application account on twitter. Creating the twitter application account is simple. Follow the steps below:

Step1: Login to your twitter account and then open the dev.twitter.com url. Go to Manage your app link on dev.twitter.com site.

Step2: Click on the create app link.

Step 3: Enter the information in the twitter create application form and press the create application button.

 Step4: Once you create the application, login to the application and change the settings, access tokens, etc.

Step5: change the permission to read and write, so that your application can download and upload twitter. You will need the access tokens in the application your are building to access your twitter account. Also, check the Allow this application to be used to Sign in with Twitter box on the settings tab.

 

Step6: Use java (twitter4j.org) or some other libraries available online to create your application that can do twitter upload, search, download .

The Java program I tried to read tweets is from this link:

"http://www.javacodegeeks.com/2011/10/java-twitter-client-with-twitter4j.html"

You need to download the twitter4j jar files from www.twitter4j.org website and set up the jar files in your eclipse before you use the java program in the link above.

Java program to add or substract dates

Java program to add or substract dates

Below is the java program for date addition. In example below, you are adding 3 days to date 20140101. You can change the format of the date by changing the format mentioned in the code in red below. 

-----dateadd.java code------------

import java.io.*;
import java.util.*;
import java.text.SimpleDateFormat;
import java.text.DateFormat;
import java.text.ParseException;
public class dateadd {
public static void main (String[] args) {
DateFormat formatter = new SimpleDateFormat("yyyymmdd");
Date dt=new Date();
try
            {
 dt = formatter.parse(args[0]);
Calendar c = Calendar.getInstance();
c.setTime(dt);
c.add(Calendar.DATE,Integer.parseInt(args[1]) );
dt = c.getTime();
System.out.println(formatter.format(dt));
            } catch (ParseException e)
            {
                e.printStackTrace();
            }
}
}


Also check out the commonly used date functions in netezza:
http://dwbitechguru.blogspot.ca/2014/09/commonly-used-netezza-date-functions.html