Getting Started with Zephir and PHP 7.2

Zephir is a new high level language that makes writing PHP extensions much easier than before, allowing one to create amazing performance enhancements by rapidly developing custom extensions for your application where processing power would otherwise be a bottleneck. But, Zephir taken a while to catch on because it is still a young project and not as easy to install as one might hope due mainly to fragmented documentation and lack of distribution packages at this point.

Zephir Installation Overview

1. Install Zephir Parser first! – The Zephir parser is a PHP extension that needs to be installed on your development machines php-cli. You must install the extension before you can install the Zephir compiler. The Zephir compiler documentation does not say this explicitly enough. There is a dependency by the Zephir compiler that requires the Zephir Parser PHP extension to be installed and working for the compiler to install and run properly. Like any extension, it is basically a matter of dropping the zephir_parser.so into the right directory, and putting an `extension=zephir_parser` directive into the right .ini file. Now, here’s the tricky part.

   When you run the install script that compiles the Zephir Parser extension and should place the .so file in the right place, but you really benefit from running the installation script with the correct flags pointing to the target version of PHP that you’ll be using for development. So, run the installation script with the following flags adapted for your version:

   sudo ./install –phpize /usr/bin/phpize7.2 –php-config /usr/bin/php-config7.2

   That’s the trick to getting Zephir Parser to compile on Ubuntu 16.04 against PHP 7.2 as of February 2018. I suspect it may be blocking potential users on other distributions as well. It is buried in the “Advanced Compilation” section of the readme, but the flags for the install script currently have no documentation and the need in some instances to compile with the –phpize and –php-config flags needs to be more prominently stated.

2. Install the Zephir compiler only after you’ve installed the PHP extension on the php-cli command line tool. Use `php -i | grep zephir` and look for evidence that the Zephir Parser extension is installed. Only then should you attempt to install the compiler.

That’s the high level process. Most everything else is obvious from the documentation. Hopefully, this overview helps at least one other PHP developer get up and running with Zephir toolkit and start learning the Zephir language, which is the fun part.

Zephir and PHP 7.2 Rule!

I’ve been working with PHP since 2000 and I still enjoy using it. And now, with the release of PHP 7.2, which has all the good parts of PHP 7, including increased performance and a better type hinting system, PHP 7.2 fixes a number of bugs that impact security, such as a broken random number generator that didn’t produce cryptographically useful random values. I think if developers begin utilizing the extension system more often through Zephir, the combination of improvements in PHP 7 and an easier path to developing language extensions, will allow the PHP community to build faster, more robust applications and the PHP platform may regain some of its respectability. Zephir and PHP 7.2 and bring new life to a development platform that deserves more respect than becoming yet another legacy language.

Advertisements

How to Hash a Long Password with bcrypt

Hashing is an important competent in securing any application that users will log into, such as a Web application or an API for an an app, particularly when authentication is done using passwords, and bcrypt is currently a standard, acceptable way to create a secure hash. bcrypt’s acceptability may change in the future, but what won’t change is the reality that no sane person stores user password values in a file or database. Let me say that again. No sane person stores user password values in a file or database. Instead, the current best practice is to store a one way cryptographic hash based on the password and later, when the user logs in, hash the password that the user supplies with the one previously stored on the system.

Beyond simple hashing of a user supplied password, there are other use cases for wanting to securely hash the entirety of an arbitrarily long string using bcrypt such as constructing authentication tokens. One may also wish to salt the password before passing it into bcrypt for an even higher level of entropy.

One algorithm I use to get around the character limit is to use an intermediate hash. So, instead of hashing the password directly, I use a weaker hashing mechanism that produces a shorter string within the length limitation of bcrypt and then hash that intermediate string using bcrypt. The intermediate hash needn’t be strong hash, like bcrypt, though if you use a highly randomized and long salt you will maximize the effectiveness of this approach. My current intermediate hash is sha512.

A Recipe for bcrypt Hashing Passwords Using and Intermediate Hash

The steps I use for hashing ginormous strings with bcrypt, such as heavily salted passwords, are as follows:

1. 1. Generate a long, cryptographically secure salt using a source such as /dev/urandom. (Do this per user. You can also append an application-wide salt as well if you like.) You will need to store this salt if you plan to do password validation in the future.
2. Concatenate the salt to the user supplied password.
3. Feed the concatenated value to sha512 and capture the result, which will be a 128 character string representing a hexidecimal number.
4. To add entropy and make use of the entire intermediate salt, loop through the entire sha512 hash taking pairs of hex digits, converting the pair’s value to an ascii character and using each character to build a compacted intermediate hash that is 64 characters long where there are 256 possible characters.
5. Feed the final compacted intermediate hash into bcrypt with your preferred parameters (such as cost) and capture the resulting hash.
6. You now have a secure hash that utilizes the entirety of the value of your long password.

This is a simple solution to the problem of limited string length support for the bcrypt hashing algorithm, but there is room for improvement. For instance, using some simple stupid math tricks, one could create variations in the length of the hashed string, which up to a minimum viable length could be used to produce a larger number of possible hash-able values. To the extent that variation and unpredictability continue to make hashed values harder to guess, adding procedural complexity to this solution is really up to the creativity of the individual implementing it.

PHP Sample bcrypt Intermediate Hash Code Example

<?php
/**
 * A simple demonstration impementation in PHP.
 * This code snippet requires PHP version 7.2 or higher. Check
 * out PHP 7.2+'s improved encryption capabilities at
 * https://php.net
 */

// This is the string that needs a one way hash. A clever user
// could make it longer than bcrypt can handle.
$password = 'spicy pony head rules';
echo "The example password is: {$password}\n\n";

// Create hypothetical cryptorgraphically secure random values for
// an application secret and a user secret. Secrets should be quite
// large so they cannot be easily guessed or brute forced.

// The application has a secret stored in a private file. It should
// be randomly generated.
$applicationSecret = base64_encode(random_bytes(2048));
echo 'Application Secret: ' . substr($applicationSecret, 0, 48) . "...\n";

// The user secret is stored with other user data. Each user gets
// their own random secret.
$userSecret = base64_encode(random_bytes(2048));
echo 'User Secret: ' . substr($userSecret, 0, 48) . "...\n";

// If you concatenate these strings, they are too long for bcrypt to use the
// entire string.
$stringToHash = $password . $userSecret . $applicationSecret;
$len = strlen($stringToHash);

echo "The hashable string is {$len} characters long.\n\n";

// So, lets use sha512 as our intermediate hash since it gives us 128 bytes
// of hexidecimal values.
$intermediateHashedString = hash('sha512', $stringToHash);
$len = strlen($intermediateHashedString);
echo "The sha512 string is {$len} characters long. SHA512:\n{$intermediateHashedString}\n\n";

// Now, let's get full use of this string by taking pairs and using
// their numeric value to create a string of one byte characters.
// Effectively, we've created a base 256 modal number that fits
// the entire intermediate hash into bcrypt's input length constraint.
// (Don't output this to your terminal. It's likely full of control
// characters).
$base256HashedString = '';
for ($i = 0; $i < $len; $i += 2) {
    $base256HashedString .= chr(hexdec(substr($intermediateHashedString, $i, 2)));
}

$len = strlen($base256HashedString);
echo "The base 256 compacted sha512 string is {$len} characters long. \n\n";


// Finally, let's serve up our bcrypt hash.
$bcryptHash = password_hash($base256HashedString, PASSWORD_BCRYPT);
echo "Out final bcrypt hash is: {$bcryptHash}\n";

Listing the Current Branches of Multiple Git Repositories Under a Singe Parent Directory Using Bash

An interesting problem came up a work where I am maintaining a couple of legacy enterprise applications written in PHP. These applications were first developed over a decade ago and the version control was originally managed in SVN. Later they were moved to Git. I do not know the full reason why, though I have a theory, but both applications break down into multiple components, each of which is managed in a separate Git repository. This isn’t a huge issue, but it does create headaches when working on multiple tickets that impact multiple components. I want to make my changes for each ticket on a new Git branch for that ticket while leaving unmodified components on their master branch in Git. Unfortunately, there isn’t an easy solution to figure out what branch each Git repository is pointed at, a problem when there are perhaps ten separate repositories that all interact with each other.

Anyhow, yesterday I wrote a Bash script to address the issue, so I can now get a quick summary of what branch each repository is currently pointed at. Since writing the Bash script took a bit of research, trial and error (as Bash does), and was generally a bit of a pain in the ass (as Bash scripting is), I figured I’d publish my Bash script so that someone else out there might benefit from what otherwise would be a single use tool written for myself, and possibly a few interested team members working with these clusters of Git repos.

---

#!/bin/bash
app_path="${HOME}/path/to/application"

# Get all the repo paths for opsdb and store them in an array.
repo_paths=()
while IFS= read -r -d $'\0'; do
    repo_paths+=("$REPLY")
done < <(find ${app_path} -maxdepth 1 -mindepth 1 -type d -print0)

#echo ${#repo_paths[@]} # DEBUG Print the legth of the arrray.

# For each array, write the path and output the current branch.
for component_path in ${repo_paths[@]}; do
    echo -n "${component_path}: "
    git --git-dir=${component_path}/.git --work-tree=${component_path} rev-parse --abbrev-ref HEAD
done

SSHFS on OS X via Fuse and the Horrible Dot Underscore Turd Files

Once again, I have been issued a Mac to do development work, and as usual everything unixish on OS X almost works. To my delight, when faced with having to code against a Linux environment that I access via SSH, I found that since I last faced this situation, someone had made and is maintaining a port of Fuse for OS X with easy support for SSHFS. This discovery was a really great way to start the morning. I downloaded the installers, ran them, wrote my script to mount my remote machine to my filesystem, and had it working by the time I finished my first cup of coffee. Kudos to the team who maintain that port!

Of course, with the OS X filesystem in the mix, network attached filesystems never just work the way you want them to. In this case, the problem I ran into was the presence of files beginning with ._ the moment I created a new file in my repository and ran git status. I’ve seen these before. They pollute the files of most every filesystem ever touched remotely by a Mac, and they are totally invisible to the Mac user. They are used to store metadata for OS X, and are totally useless for any other computer. One Stack Overflow commenter deemed these dot underscore files “turd files”, which I consider an apt description. Apple calls the double underscore files “Apple Double” files. Whatever you want to call them, when being introduced into a client’s code repository on a Linux OS, these files beginning with ._ are turds and I want to prevent them from ever being created on my remote system.

It took some digging using a brand-name search engine, the name of which I will not disclose, but I was eventually directed back to the OSXFuse documentation, which, with a little reading, gave me a solution. (Cheers to working documentation!)

In the SSHFS connection string there are several options that can impact the creation of Apple Double (._) files. The two that I wound up using were noapplexattr and noappledouble. noapplexattr prevents OSX from even attempting to apply its extended attribute shenanigans to the SSHFS mounted operating system and noappledouble explicitly prevents the creation of any file beginning with a dot underscore as well as the also very annoying .DS_Store file. I’m not certain that both options were needed for my situation, but I used both and tested the result, and it seems to be working. Not almost working, just working.

My command for mounting my SSHFS looks like this (with sensitive bits redacted, of course):

sudo sshfs -o allow_other,defer_permissions,noapplexattr,noappledouble,IdentityFile=~/.ssh/id_rsa myusername@clientsserverhostname:/home/myusername /mnt/sshfs/mymountpoint

So long ._ files!

Server-Side Swift on Linux

“Can I write Web applications in Swift?” was a random question that popped into my head a few weeks ago after discovering Swift.org, a project that moves Swift programming language out of the domain specific role of replacing Objective C as the defacto language for building apps targeting Apple’s iOS and OS X platforms and into the Open Source ecosystem. The answer is, yes, thanks to the work being done by Perfect.

I have been curious about Swift since it first came out, though I am not a mobile developer. Recently, in a job interview, I spoke to the mobile team at a digital media company, and the iOS developer raved about the language, which led me down the rabbit hole. Seeing that Swift is capable of being a general purpose programming language, and that it can run on Linux makes me optimistic that organizations with strong iOS offerings may be able to run full-stack Swift applications similar to what Node.js / Web developers currently enjoy. The core advantage of Swift, of course, being not JavaScript. Whether Swift becomes a mainstream server-side language outside of the Apple fanpeople’s realm is anyone’s guess, but it is intriguing enough that I will be giving it a try over the next couple of weeks.

Today’s effort was seeing if I could get Swift and Perfect up and running on my Ubuntu 16.04 machine. Swift.org offers binary downloads for Ubuntu 14 and 15, but not 16, as of today. However, being lazy (or arguably organized) and wanting to used apt to manage the installation, I instead found this Swift package for Ubuntu 16.04, which, following the instructions, worked flawlessly, though it did involve editing my .bashrc file to include the package’s binary directory in my user’s PATH variable.

After installing the Swift compiler, I downloaded the Perfect sample project and built it, as per Perfect’s instructions. My first attempt at compilation failed, as I received the error:

/home/david/swift/PerfectTemplate/PerfectTemplate/Packages/LinuxBridge-2.0.1/LinuxBridge/include/LinuxBridge.h:6:10: fatal error: 'uuid/uuid.h' file not found
#include <uuid/uuid.h>
^
1 error generated.
<unknown>:0: error: build had 1 command failures

This seems to be a common dependency problem, and after a little digging, I was able to get the compilation to complete by running the command sudo apt-get install uuid-dev to install the uuid development package for Ubuntu. Easy peasy, right?

Great success!

The next step for me will be to try and build a working Web app on Linux in Swift using the Perfect library. Whether I find the language as pleasant as some iOS developers remains to be seen, but it seems like it is an experiment worth trying.

 

Boycott San Francisco!

People love San Francisco. There’s the great food, the proximity to wine country, and a general lack of snow. San Francisco has a rich heritage of counter culture, dreamers, homosexual poets, hippies, hipsters, and yuppies. Yes, I said it: yuppies. Not that yuppies are inherently bad. Okay, yuppies are inherently bad, but having a few bad apples around helps remind you how good the good apples really are. Having a lot of bad apples, though, means that you just have a lot of bad apples.

I’ve lived in the Bay Area for almost 30 years, San Francisco for six, and never before have I seen things get so bad and desperate. In a time when most regional US economies are not producing enough jobs, the Bay Area seems to have done the opposite. We have overproduced jobs to the point where there is no more housing for all the people who are coming here to fill those jobs. It has gotten so bad that I have begun looking for jobs elsewhere, in places where it might snow even, because I cannot afford to live here anymore. But I can’t find jobs in tech, outside of the few core hubs, that pay a competitive salary or match my level of skills and seniority. What I do find on sites like Craigslist are companies posting in other regions ads that beg developers to COME TO SAN FRANCISCO.

Well, here’s a news flash for all those employers. About fifteen years ago broadband Internet became a thing for middle class consumers. You do not need to bring any more warm bodies into your offices to create synergy. Those warm coding bodies would be happy to take 80% of what you’re offering an on-site employee and quietly code at home in Denver or Minneapolis or damn near anywhere else you can think of. And they will be more efficient, because the last thing you ever want your software engineer doing when they’re hacking on your core product offering is worrying about the increasingly realistic possibility that they may be homeless while earning a low six-figure salary, because a low six figure salary will not pay for a modest apartment in San Francisco these days.

And if you’re a programmer on the outside of Babylon looking in, dreaming of taco trucks, cheap cabernet, and enlightened free spiritedness, think twice. The people who are making what elsewhere would be very generous salaries by moving here are subsidizing their lifestyles. They’re going into debt, living off of trust funds, or growing weed in their closet. No one here has a retirement fund.

Maybe this is where the VCs are. Maybe this is where the intellectual ferment of the digital age starts, but it doesn’t have to end at the edges of the Bay, or even at the city limits of (gasp) Oakland. You can take it home. You can tell that VC that you’re going to start your company in your home town and save tons of their capital in the process. You can create jobs in your community. And if all you want is a job, you can tell Robollox or whomever is poaching for talent in your community that you would love to join their development team and you’d be happy to fly in once a month for the requisite face time, but you would rather rot in Hades than to give all of what should be your discretionary income to a scum sucking absentee landlord in the most screwed up housing market in these United States of America. Please, take those companies jobs and money, but also, please don’t come here. There is nowhere to live here. There are people renting tents in their back yards for $1,000 a month. There are people renting those tents. You will pretend to have fun for a while even though you will have no time or money to enjoy this once fabulous city. Then you will get sick of all of it and leave. The actually fun people can’t afford to live here anymore. The ones left, the odd indigenous San Franciscan excepted, are the die hard core of uptight yuppie scum who enjoy the smell of their own farts and will pay infinite amounts of rent for the status of having the correct zip code.

Move here at your own peril. I recommend an all out boycott. San Francisco has ceased to be livable and no one can afford flowers to wear in their hair.

URL Rewriting via VirtualHost Directive for the Slim PHP Framework

I’ve been using the Slim framework on and off for the past couple of years. I find it to be a nice piece of glue to structure my PHP applications without having to buy into someone else’s entire toolkit. In fact, I use it because it lets me use whatever tools I want and works seamlessly with Composer.

One thing that I find problematic getting a Slim project up and running is the insistence of the documentation that the rewrite to the front controller on Apache be configured in an .htaccess file. This is fine for shared hosting environments, but on applications where I control the server configuration and want to harden my infrastructure by avoiding such pitfalls as putting my server configuration in my document root, I would rather not have to put AllowOverride All anywhere in my Apache configuration files.

So, without further discussion, here is a VirtualHost file that I used to get Slim up an running on Apache with no .htaccess file. This was done for Apache/2.4.7 running on Ubuntu 14.04LTS. It took me several hours to remember how to do this, so hopefully this saves someone else some time in the future.

<VirtualHost *:80>
        ServerName web01.dev.example.com

        ServerAdmin david.cloutman@example.com
        DocumentRoot /var/www/web01.dev.example.com/htdocs

        RewriteEngine Off

        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteRule ^ /index.php [QSA,L]

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined


        <Directory />
                RewriteEngine On
                RewriteCond %{REQUEST_FILENAME} -s [OR]
                RewriteCond %{REQUEST_FILENAME} -l [OR]
                RewriteCond %{REQUEST_FILENAME} -d
                RewriteRule ^.*$ - [NC,L]
                RewriteRule ^.*$ /index.php [NC,L]
        </Directory>
</VirtualHost>