Creating Your Own SSL Certificates For FileMaker

Typically, you buy an SSL certificate for a server from a SSL vendor. However, some companies may decide that they want to issue their own SSL certificates. Often this is because the domain is only used internally, and most vendors don’t easily allow (if at all) the signing of server certificates for non-public domains. Additionally, issuing your own certificates can remove complications caused by the certificate verification process used by most vendors, and there are no fees needed for each certificate.

Although this use case is not supported by FileMaker, I had a client who had no choice but to use their internally signed certificates, and was having issues getting to work on their newly updated server, which had been working fine with FileMaker Server 15. Since I didn’t have access to the server (their security protocols did not allow that), I decided to go through the process of creating my own CA (certificate authority), using it to issue an SSL certificate, and see if I could install it on a test instance of a FileMaker 16 Server.

First step was to create the certificate authority, which I did on a separate macOS system using its Certificate Assistant.

Then started filling out the screens. After the first screen most were left at their default values.

Once done with the CA, I again used the certificate assistant to create a server SSL certificate using the beezwax.int CA.

With that done, I then exported both certificates in p12 format and copied them to my workstation.

In order to load them on to FileMaker Server however, I first had to convert them from .p12 format to .pem format. I used the OpenSSL command to export both the beezwax.int and the fmtest.beezwax.int certificates.

$ openssl pkcs12 -in /Volumes/data/Users/sibr/Desktop/fmtest.beezwax.net.p12 -clcerts -nokeys -out fmtest.beezwax.int.pem
Enter Import Password:
MAC verified OK

For the fmtest certificate I also need the key file, so we use a slightly different version for that.

$ openssl pkcs12 -in /Volumes/data/Users/sibr/Desktop/fmtest.beezwax.net.p12  -nocerts -out fmtest.beezwax.int.key.pem
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:

With that, I now imported them in to the FileMaker server. This was a FileMaker Server 16.0v3 system running Windows Server 2016 Standard. Notice that I used the CA certificate as the intermediate certificate in the import dialog.

I then restarted the FileMaker Server service. Back on my workstatation, I imported the beezwax.int root CA and marked it as trusted for all certificate types. You’d probably want to be more specific.

In order to quickly spoof the hostname without creating actual DNS records I’ve modified my /etc/hosts file so that my workstation will resolve the fmtest hostname to the test server.

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost
255.255.255.255 broadcasthost
::1             localhost
fe80::1%lo0     localhost

192.168.10.76   fmtest.beezwax.int

I found that I could now go to the site using the hostname with Safari and Chrome and get a SSL lock icon and the expected certificate chain. Here is what it looked like in Safari.

Firefox however did not work. This is at least in part because it manages certificates separately from the OS, so the beezwax.int root certificate was not present when it tried to validate the fmtest certificate. However, importing the beezwax.int certificate at Firefox’s Authorities tab in Certificate Manager wasn’t sufficient, so this remains an outstanding issue.

Possibly more important than Firefox, depending on your usage, was how FileMaker Pro responded. It was strangely conflicted. On one hand, it gave me a warning about being unable to verify the certificate, but when I viewed the certificate information, it indicated a valid certificate.

After a bit of digging I found the location for the root CA files in FileMaker Pro and added in my CA certificate there.

I was then able to connect to the server without warnings with FileMaker Pro.

Simon