Tech Nugget

Sunday, June 22, 2014

Apex Data Loader in Linux

Salesforce Apex Data Loader is only officially supported on the Windows platform but the source code is available as an open source project for anyone to build it for another platform such as Linux or Mac. The data loader is written in Java using the Spring Framework so it can be easily rebuilt for any platform that supports the JDK and corresponding Eclipse SWT for GUI. If you don't care much for the GUI to do the mapping and simply want to use it from the command-line, then the good news is that the same JAR file from the Windows installation can be used to run the process in another platform without having to rebuild the code.

What's missing however are the Linux version of the utility scripts (in the bin\ directory in windows installation) to do the password encryption and process execution. I recently had to execute the data loader jobs from Linux and ended up writing the missing scripts so it is easy to mimic the Windows command line process (as explained in Chapter 5 in the guide).

The code is on github and includes the default configuration options and a sample extract process. The directories mirror the Windows version and the following steps walk you through the process:

From your Linux (or Mac) terminal, execute the following command to clone the project from github
$ git clone https://github.com/sthiyaga/dataloader.git
Copy the dataloader-30.0.0-uber.jar from the Windows installation to dataloader/ directory (from above)
Generate the private key to encrypt the password
$ bin/encrypt.sh -g <some-random-seed-text>
Copy the output from Step 3 above to the file conf/private.key (replacing the text in there)
Encrypt the salesforce password (+security token, if required) using the generated private key
$ bin/encrypt.sh -e "password+securitytoken" conf/private.key
Copy the output from Step 4 above to the conf/config.properties file for the sfdc.password token value
Update the conf/config.properties file with sfdc.username and sfdc.endpoint token values
Optionally, adjust any other default parameters in the conf/config.properties file
Run the sample account extract process
$ bin/process.sh csvAccountExtractProcess

That's it, you should have the sample extract in the data/ directory.

Enjoy and feel free to use the code!

-Senthil

Friday, June 6, 2014

Java Encryption, Apex Decryption

Ever had a need to decrypt something in Apex that was encrypted in an external app? Well, I just ran into a need for it where an external Java web app generates a token that is encrypted with AES and when it is sent to Salesforce.com, the token had to be decrypted to do a bunch of stuff. There are a couple of things to note if you have a similar need -- Salesforce supports encryption/decryption using AES algorithm (128 bits, 192 bits and 256 bits) but is particular about the encryption mode and padding. The mode has to be cipher block chaining (CBC) and the padding has to be PKCS5. These are different from what Java supports by default, so it is important to pass the correct settings to initialize the cipher in the encrypting application. Secondly, the initialization vector (IV) used during AES encryption can be provided separately or as part of the encrypted text and the appropriate method in the Crypto class in Apex should be used for decrypting. Refer to the docs for the Crypto class for all the gory details - http://www.salesforce.com/us/developer/docs/apexcode/Content/apex_classes_restful_crypto.htm

Ok, now here is a sample class in Java for generating the AES 128-bit key that will be used by both the web application to encrypt and the Apex code to decrypt the token:

import javax.crypto.KeyGenerator;

import javax.crypto.SecretKey;

import javax.xml.bind.DatatypeConverter;

public class AESKeyGen {

 public static String generatePrivateKey() throws Exception {

  KeyGenerator keyGen = KeyGenerator.getInstance("AES");

  keyGen.init(128);

  SecretKey secretKey = keyGen.generateKey();

  return DatatypeConverter.printBase64Binary(secretKey.getEncoded()); 

}

 public static void main(String[] args) throws Exception {

  String privateKey = generatePrivateKey();

  System.out.println("Private Key = " + privateKey);

}

}

When run, this will produce a Base64 encoded AES private key that is 128-bits long. Needless to say, this should be kept private from the prying eyes.

Private Key = useALumM/MAmHU1+hgsnPg==

Next is the Java class that actually encrypts the given text using this private key. In this case I'm including the IV data as part of the encrypted text by prefixing it before encoding to Base64 format.

import java.security.SecureRandom;

import javax.crypto.Cipher;

import javax.crypto.spec.IvParameterSpec;

import javax.crypto.spec.SecretKeySpec;

import javax.xml.bind.DatatypeConverter;

public class AESEncrypt {

 public static String encodeAES(String privateKey, String data) throws Exception {  

  final int AES_KEYLENGTH = 128; 

  byte[] iv = new byte[AES_KEYLENGTH / 8];

  SecureRandom prng = new SecureRandom();

  prng.nextBytes(iv);

  Cipher aesCipherForEncryption = Cipher.getInstance("AES/CBC/PKCS5PADDING");

  aesCipherForEncryption.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(DatatypeConverter.parseBase64Binary(privateKey), "AES"), new IvParameterSpec(iv));

  byte[] byteDataToEncrypt = data.getBytes();

  byte[] byteCipherText = aesCipherForEncryption.doFinal(byteDataToEncrypt);

  byte[] ivPlusCipher = new byte[iv.length + byteCipherText.length];

  System.arraycopy(iv, 0, ivPlusCipher, 0, iv.length);

  System.arraycopy(byteCipherText, 0, ivPlusCipher, iv.length, byteCipherText.length);

  return DatatypeConverter.printBase64Binary(ivPlusCipher);

}

 public static void main(String[] args) throws Exception {

  if (args.length < 2) {

   System.out.println("Usage: java AESEncrypt <private-key> <text>");

   System.exit(-1);

}

  System.out.println("Cipher Text = " + encodeAES(args[0], args[1]));

}

}

Here is a sample execution of the encryption class with the above private key:

> java AESEncrypt "useALumM/MAmHU1+hgsnPg==" "Hello, (Encrypted) World!"

The output should be something similar to what you see below (it changes with each run as the IV is generated randomly each time) and again, the output includes the IV + Encrypted text in Base64 encoded format.

Cipher Text = 86z+0A1LKPluwKEmeczUqMYMCLo6BlEOeHwx6zZh/bfsrXR8oRg2Z+csM9UHL59J

Now that we have our private key and encrypted text, the following lines of code will decrypt it in Apex (run through the Developer Console in this case) -- note the use of decryptWithManagedIV method since the encrypted text includes the IV in our case:

Blob key = EncodingUtil.base64Decode('useALumM/MAmHU1+hgsnPg==');

Blob encData = EncodingUtil.base64Decode('86z+0A1LKPluwKEmeczUqMYMCLo6BlEOeHwx6zZh/bfsrXR8oRg2Z+csM9UHL59J');

Blob decryptedData = Crypto.decryptWithManagedIV('AES128', key, encData);

System.Debug( decryptedData.toString() );

Here is the output (from the debug log):

18:16:50:025 USER_DEBUG [6]|DEBUG|Hello, (Encrypted) World!

That's it, feel free to reuse the code in your projects as needed. Happy Friday!

-Senthil

Thursday, December 5, 2013

Command line access to force.com #2

In the previous post I showed how to use cURL to login to salesforce.com and execute a query. It required creating a Connected App so you can use OAuth2 to authenticate and authorize. I was immediately asked by a few folks around here about the need to do that and if there was a simpler way to do this. So the good news is, yes, you can use the SOAP API to login and then use the session Id as the OAuth2 token -- it's perfectly fine.

Here is how you would issue a SOAP login request to salesforce.com using cURL without the need to configure anything:

curl -X POST https://test.salesforce.com/services/Soap/u/29.0
-H "Content-Type:text/xml"
-H "SOAPAction: login"
-d "<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:urn=\"urn:partner.soap.sforce.com\"><soapenv:Header/><soapenv:Body><urn:login><urn:username>YOUR_USERNAME_HERE</urn:username><urn:password>YOUR_PASSWORD_PLUS_SECURITY_TOKEN_HERE</urn:password></urn:login></soapenv:Body></soapenv:Envelope>"

Note that the above command must be in a single line.

Now you should get a response back in XML with a bunch of details and among them the key fields to grab are the serverUrl and sessionId. The serverUrl will be of the form: https://XYZ.cs12.my.salesforce.com/services/Soap/u/29.0/00DV000000YYYYY
Just use the hostname part of the url and append /services/data/v29.0/query?q=your_query along with the sessionId as the Bearer token and you are good to go. The same query command as in the previous post should work fine with these values:

curl https://XYZ.cs12.my.salesforce.com/services/data/v29.0/query?q=Select+Name+From+Account+Limit+5 -H "Authorization: Bearer 00DV000000XXXXX!ARsAQNuDPT9Bhz9FPQFf.DZEUqATv7qbBIFc60YBOy3dvQEVEGYq4Q6iO379NYL5oeWW5yWGeBMfvrUYtyoYYYYYYY.PYFK"

The results are returned in JSON format, exactly like before!

Have fun!

Command line access to force.com

If you come from a Unix background you will surely miss the good ol' days of piping input and output between the different commands and seeing magic happen. In today's cloud connected world most of us end up working with web services and hardly do any command line anymore. There are still perfectly good use cases for using them though, especially for scripting and automating a few steps and in the case of force.com, doing deployments.

A few days ago I saw a post on developer.force.com blogs about the command line interface to force.com and I got all excited to try it out. It worked great for most part and I haven't played around with all the commands or even enough to script out a flow for specific use cases. However, the one thing that did catch me by surprise was the way authentication was done -- it uses a local HTTP server and connects to a web app hosted in heroku -- I didn't fully understand what was going on and I'm not a security expert to weigh in on the merits of this solution but I was wondering why it couldn't be done simply using OAuth2 Username/Password flow. I would have to brush up on Go programming language to perhaps extend the actual code (gladly, it's open source and on github, thank you!) but in the mean time I wanted to show how to authenticate and execute commands from the command line using cURL.

You would have to setup a Connected App as explained here. Once you set it up, you would have a Consumer Key and Consumer Secret. The next step is to get OAuth2 Access Token and for that issue the following cURL command (use test.salesforce.com for sandbox and login.salesforce.com for production or developer edition) substituting the appropriate values:

curl -X POST https://test.salesforce.com/services/oauth2/token
-d "grant_type=password"
-d "client_id=your_consumer_key"
-d "client_secret=your_consumer_secret"
-d "username=your_username"
-d "password=your_password_plus_security_token"

Note that the above command must be issued in a single line and is shown in separate lines for clarity.

Now SFDC should authenticate, authorize and provide the response in JSON format as follows (actual values changed for obvious reasons):

{
"id":"https://test.salesforce.com/id/00DV00000087XXXXXX/00530000007stYYYYY",
"issued_at":"1386263432924",
"instance_url":"https://XYZ.cs12.my.salesforce.com",
"signature":"WLituc1zj2tSwN7F77p1LY8slJBbGtE2kWb1mJ9H12k=",
"access_token":"00DV00000087XXX!ARsAQNsi.0NW.mgEzP8ok_Pb.dQVam6x0ZS.1GQkjSfkTF5K"
}

The two key pieces of information that we need from the response are the instance_url and the access_token. The instance_url provides the actual sandbox (or production) instance to use and the access_token is the Session ID. Now you can make subsequent calls as follows:

curl https://XYZ.cs12.my.salesforce.com/services/data/v29.0/query?q=Select+Name+From+Account+Limit+5 -H "Authorization: Bearer 00DV00000087XXX!ARsAQNsi.0NW.mgEzP8ok_Pb.dQVam6x0ZS.1GQkjSfkTF5K"

The Authorization header should include the access_token as the Bearer token. This request should now return the first five accounts in JSON format as follows:

{

"totalSize":5,

"done":true,

"records":[

{