Convert JKS file to PKCS12 to connect Sitecore and Salesforce

Salesforce uses JKS files to store information about the valid certificates used for API communication. The challenge is that JKS is not a format very common in the .net applications – we more often use certs in PKCS12 format.

How to convert JKS file to PKCS12 file?

Before you start conversion it is good to verify if JKS file contains only one certificate – the one you need. To verify what is inside the JKS file you can use “KeyStore Explorer” application that you can download here.

When you open the JKS file with KeyStore Explorer you will see something like this:

When you are sure that that JKS file contains only one cert you can start conversion. For conversion, you will need access to the keytool that usually is available in the OpenJDK directory.

In my case the keytool is available in the path:
C:\Program Files\ojdkbuild\java-1.8.0-openjdk-1.8.0.232-1\bin\keytool.exe

When you find the keytool you can run the following command to convert jks into pkcs12 file:

C:\Program Files\ojdkbuild\java-1.8.0-openjdk-1.8.0.232-1\bin\keytool.exe' -importkeystore -srckeystore xxxxxxxxxxxxxx.jks -destkeystore xxxxxxxxxxxxxx.p12 -srcstoretype JKS -deststoretype PKCS12

Just replace “xxxxxxxxxxxxxx” file name of you jks file and the name of the p12 that should be generated.

If everything went fine you can now use your cert in a new format to sign data that you send to Salesforce. More info about that you will find in the article “Sitecore and Salesforce custom integration – Salesforce JWT bearer token generation

Sitecore and Salesforce custom integration – Salesforce JWT bearer token generation

In this post I will explain my approach to the Salesforce – Sitecore integration. In fact code that is going to be described here can be used in almost every .net application.

Available Sitecore – Salesforce integration

Sitecore provides an integration to everyone who needs to synchronize the data between the Sitecore and Salesforce – the “Salesforce Connect” extension.

You can download it here: https://dev.sitecore.net/Downloads/Salesforce_Connect.aspx

If you do not know which version you should install, check it here (comatibility table): https://support.sitecore.com/kb?id=kb_article_view&sysparm_article=KB1000576

The natural question that probably comes to your mind right now is ‘why did not I use it when I had a need to connect to salesforce’ – this is a really good question.

The answer is – because available extension is focused on the syncing contacts between Salesforce and Sitecore, when in our case we just wanted to send some data to Salesforce and the synchronization was being made on the different level.

In other words, available extension did not meet our needs.

Few general words about the integration

Integration with Salesforce is not in any kind special – it is just an API that is managed by the Salesforce developers. An endpoints and parameters can differ but the common thing is token generation that I decided to describe in this post because it took me a while to understand how to generate the correct JWT bearer token that will be honored by the Salesforce endpoint.

Documentation about it is available on the Salesforce help portal : https://help.salesforce.com/s/articleView?id=sf.remoteaccess_oauth_jwt_flow.htm&type=5

Documentation contains the Java code that I had to transform into .NET one what sometimes was not so obvious.

What do you need to generate OAuth 2.0 JWT bearer token

To make communication possible you must have generated certificate that is going to be used by Salesforce and your Sitecore instance. When certificate will be installed on the Salesforce side you can continute configuration on your side.

To generate the JWT token you need to gather the following information:

  • iss – this is OAuth client_id (provided by Salesforce)
  • aud – this is authorization server’s url (login.salesfroce.com for production and test.salesforce.com for test environments – provided by Salesforce)
  • sub – the username of account used to connect to salesforce (usually email – provided by Salesforce)
  • exp – timestamp of the expiration (provided by Salesforce)

Generation process

All of that data need to be later encoded to base64 string:

private string GenerateClaimsString()
{
var iss = this._salesforceConfigurationService.GetJwtClaimsIss();
var sub = this._salesforceConfigurationService.GetJwtClaimsSub();
var aud = this._salesforceConfigurationService.GetJwtClaimsAud();
var exp = this._salesforceConfigurationService.GetJwtClaimsExp();
var claims = $"{{\"iss\": \"{iss}\", \"sub\": \"{sub}\", \"aud\": \"{aud}\", \"exp\": \"{exp}\"}}";
return this.Base64Encoder(claims);
}

But it is not only encoding by the standard Convert.ToBase64String method. We must also remove some of the chars from the generated string:

private string Base64Encoder(string valueToEncode)
{
byte[] valueToEncodeAsBytes = System.Text.Encoding.UTF8.GetBytes(valueToEncode);
return Convert.ToBase64String(valueToEncodeAsBytes).TrimEnd('=').Replace('+', '-').Replace('/', '_');
}

When the values are encoded and unwanted chars are removed from the encoded string we need to add to it predefined JWT header. Header has a very similar structure to the claims and can be hardcoded with value:

"{\"alg\":\"RS256\"}"

Here is the code that can do that:

private string GenerateJwtHeaderString()
{
var headerValue = SalesforceConstants.Api.Values.Header;
return this.Base64Encoder(headerValue);
}

After all operations we have two strings that we can use to build the assertion used later in the authorization request.

var assertion = this.GenerateJwtHeaderString();
assertion += ".";
assertion += this.GenerateClaimsString();

As you can see two strings are again connected with the dot sign.

But this is not the end – now we are going to use our certificate to sign the assertion. Full code of assertion generation will look like this:

var assertion = this.GenerateJwtHeaderString();
assertion += ".";
assertion += this.GenerateClaimsString();
this._assertion = assertion + "." + this.SignAndGeneratePayloadString(assertion);

Where SignAndGeneratePayloadString method looks like this:

private string SignAndGeneratePayloadString(string payload)
{
            X509Certificate2 certificate = new X509Certificate2(this._salesforceConfigurationService.GetCertPath(), this._salesforceConfigurationService.GetCertPass(), X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet | X509KeyStorageFlags.Exportable);
            using (var privateKey = certificate.GetRSAPrivateKey())
            {
                var signedData = privateKey.SignData(System.Text.Encoding.UTF8.GetBytes(payload), HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
                return Convert.ToBase64String(signedData).TrimEnd('=').Replace('+', '-').Replace('/', '_');
            }
}

As you may noticed SignAndGeneratePayloadString method uses certificate to ‘sign’ the assertion data – to make it work you need to load the certificate from the disk (it must be p12 certificate file) and have certificate password to read it.

The result of signing is again concatenated with the dot and original assertion data.

When the assertion is finally ready, you can authenticate with Salesforce API and Request Access Token for further Salesforce communication.

Summary

If you compare implementation from this blog post with implementation from the Salesforce’s help page you will notice few major differences like:

  • usage of p12 certificate file instead of jks
  • additional string operations on the generated/encoded string values

If you want you can check the full implementation here: https://github.com/lskowronski/SitecoreSalesforce/blob/main/SalesforceJwtService.cs

Because transition from JKS file to p12 file can be also tricky, I will describe it in the another blog post that is going to be published soon – stay tuned!

Low performance of Sitecore Experience Editor – SC 10

One of the customers had Sitecore 10 initial release installed on Production – everything was working fine till they started adding many new languages and visibility rules to the content they create – in other words they have started to use Sitecore XP in full scale, with personalization etc.

At the beginning everything was working fast and smothly but after few more languages and few more rules added, everything slowed down and editors had to wait aroud 30-60 seconds to load Sitecore Experience Editor and sometimes more complex pages were even able to stop the IIS.

There are at least few well known tweaks that can be applied to improve Experience Editor performance, majority of them are listed on the Sitecore.StackExchange : https://sitecore.stackexchange.com/questions/10627/experience-editor-painfully-slow

I had an ‘opportunity’ to found another case that later was identified by Sitecore Support as known issue with fix number: SC Hotfix 530937-1 (please contact Sitecore Support to get code of it).

Explanation of the issue

Sitecore has very advanced mechanisms checking what exactly should be shown to the visitor and when. Some of the checks are:

  • check of the date and time when component should be shown
  • check of the personalization rules when component should be shown
  • check of user permissions to display/change every component

As you see I emphasize ‘component’ in every aboves point – I do that because complex pages with many components have to run those checks so many more times to show you the page that it significantly increase time of page loading in the experience editor – where you do not have cache enabled.

I was debugging for a while and found this tool as the source of low performance:

It allows to change the date and verify how the website would looks like for certain date – with all applied rules/filters.

The tool seems to use Sitecore.Pipelines.FilterItem.GetPublishedVersionOfItem, Sitecore Kernel processor to prepare data to display it in the experience editor.

I replaced this processor with my one to verify how many times it is called and what operations are being made there and I was shocked with hundreds of requests and checks that are triggered by this processor.

I decided to remove it from the pipeline with code:

<processor type="Sitecore.Pipelines.FilterItem.GetPublishedVersionOfItem, Sitecore Kernel" resolve="true">
    <patch:delete />
</processor>

And after the patch was applied I immediately noticed that Experience Editor has started to load quickly again (I mean as fast as it is possible for Experience Editor – around 5s).

So, if you also experience very low Experience Editor performance you can try to apply that change and check if it helped – if yes, then I would suggest to contact Sitecore Support and get the official patch.

Database shards in Sitecore – how to get the data from them?

Based on the title – the topic seems to be pretty simple as Sitecore has prepared really great documentation about the shards but in this article I will try to describe scenario when we want to get the data from shards with direct connection to the database instead of using Sitecore’s libraries like xConnect etc.

As an example I will show you how to get the latests interactions saved into database. Usually to do that you need to know the contact id and query for the data via xConnect but not this time. I will explain everything very soon.

Documetation

Sitecore team has prepared few documetation pages describing how the data from shards is consumed.

Take a look at this article to get the overview od how the data is taken from each of the shards: https://doc.sitecore.com/en/developers/90/sitecore-experience-platform/data-extraction.html
You should definietly check it because it will allow you to understand later why the number of results can be multiplied by the number of the shards.

Take a look at this article to see how to get the contacts and their interactions from the shards:
https://doc.sitecore.com/en/developers/90/sitecore-experience-platform/extracting-contacts-and-interactions.html
Under that link you will notice that to get any interactions you must have a contact reference.

The challenge

Now, imagine that you do not have contact id or you just do not care about the contact data because you want to get the latest interactions that happened with the website – like goals or page visits. How to get that data without having contact’s reference?

The solution

To start we must use database connection – in our case it will be connection to the xdb collection database.

Then to work with the shards we are going to use two libraries:

  • Microsoft.Azure.SqlDatabase.ElasticScale.Query
  • Microsoft.Azure.SqlDatabase.ElasticScale.ShardManagement

These libraries have been developed for Azure but they still perfectly work on every local environment with shards.

Frist of all we must load the shards :

ShardMap = myShardMap;
var shardMapManager = ShardMapManagerFactory.GetSqlShardMapManager(...your connection string..., ShardMapManagerLoadPolicy.Lazy);
shardMapManager.TryGetShardMap("ContactIdShardMap", out myShardMap)

With that code we are loading shards configuration into the ‘map’ object.

Sitecore comes out of the box with three shards:

  • Device Profile Shard (DeviceProfileIdShardMap)
  • Contact Shard (ContactIdShardMap)
  • Contact Identifiers Shard (ContactIdentifiersIndexShardMap)

More about Sitecore’s shards you can check for instance here: https://doc.sitecore.com/en/developers/93/platform-administration-and-architecture/split-or-merge-xdb-collection-database-shards.html

When we have loaded shard map we can start query for the data – to connect to shard we must use sql connection string builder and then with that builder we can connect to the ‘part’s of the shard’.

var builder = new SqlConnectionStringBuilder(...your connection string...);
using (var conn = new MultiShardConnection(myShardMap.GetShards(), $"User ID={builder.UserID};Password={builder.Password};"))
{
    using (MultiShardCommand cmd = conn.CreateCommand())
    {
        cmd.CommandText = "SELECT * FROM [xdb_collection].[Interactions]";
        cmd.CommandType = CommandType.Text;
        cmd.ExecutionOptions = MultiShardExecutionOptions.IncludeShardNameColumn;
        cmd.ExecutionPolicy = MultiShardExecutionPolicy.PartialResults;
        using (MultiShardDataReader sdr = cmd.ExecuteReader()) 
        {
            while (sdr.Read())
            {
                var interactionId = sdr.GetGuid(0);
                var lastModified = sdr.GetDateTime(1);
                .........
                var campaignId = (sdr[10] as Guid?).Equals(null) ? null sdr.GetFieldValue<Guid?>(10);
                .....
            }
        }
    }
}

Why did I write about the parts of the shard when there is no loop iterating through the shards? Well, all that happens inside the Azure’s library. It means that depends on the number of the parts we will call the same query as many times as many shards we have. Because of that it will be much harder to maintain the number of results that we want to receive.

If we want to limit our results to 100 in fact the final result can be 200 because limit of 100 will be applied for every query called to every shard part. Despite that – it is just like a normal SQL query and it can be very helpful from time to time.

Summary

In general it is worth to member that this solution belongs to the ‘hacks’ category and if it is not required it is much better to use Sitecore’s library to work with xConnect models in the safe and well documented way. If you already tried to use it and it is not something that solves your issues then I hope that the concept of accessing database shards will save your life like it saved my – at least once so far. Good luck!

What to use to write Unit Tests for Sitecore?

In this post, I would like to share with you some experiences I had in my last project and give you advice on what you can use to write Unit Tests and why sometimes some of the libraries are not a good way to go.

In the most cases when the team is thinking to start writing unit tests there is an discussion about different approaches and available libraries – been there done that.

Let me tell you my story

In our case, we had the following libraries taken into consideration:

  • NUnit vs XUnit
  • NSubstitute vs Moq
  • Asserts vs FluentAssertions
  • FakeDB

We totally forgot about AutoFixture and that was our mistake!

After couple of meetings (we had several teams that had to agree something) we decided to vote and the winners were:

  • NUnit
  • Moq
  • FluentAssertions
  • FakeDB

Of course, you may ask “why” … the answer is usually simple because team members knew these three or just liked to write unit tests with the support of these libraries than others.

Is this logical – in some way yes. Is it professional – some people may say ‘no’ but this is how real-world works. Software developers tend to use libraries that they know – especially when it comes to writing unit tests – even when there are better solutions out there.

Everything was fine at the beginning…

We started to write tests, we had around 100 test cases and then we realized that we need something to generate fake Sitecore items automatically.

We started to play we AutoFixture library and then we realized that we made a mistake …

AutoFixture was working really good with all of the libraries instead of FakeDB. We were able to generate simple types like strings etc. but no items were added to the FakeDB.

After some digging in google I found out that many people had that issue and there are two repeated solutions:

  • switch into XUnit
  • get rid of FakeDB and prepare your own fake objects

For us was too late – we decided to stay with our setup.

What to use – the answer

Based on my experience I would not start writing unit tests for Sitecore with NUnit in the future.

The rest of the listed libraries seem to be pretty safe and good from my point of view.

So the answer is:

Use XUnit and whatever else you want – but not NUnit because it can block you at some point as blocked me.