The #1 Mistake Technical Recruiters are Still Making in 2018

Technical recruiting is hard. There aren’t enough unemployed software engineers to fill your requirements. You have to get passive software engineering candidates in front of clients. As a result, many of your candidates are not actively looking for work. So why do you expect them to have a current resume, especially one specifically in Microsoft Word format?

Read More »

Advertisements

Installing Zephir with PHP 7.2

For any PHP developer wanting rapidly develop PHP extensions or optimize their PHP application, installing Zephir is a good first step towards eliminating their PHP application’s execution bottlenecks. While installation of Zephir is not terribly complicated, installing Zephir on Linux systems (such as Ubuntu) or on OS X is much easier when you understand Zephir’s architecture and how the compiler integrates with the PHP interpreter. There are two top-level steps in the installation procedure which must be done in the right order.

Zephir is a new high level language that makes writing PHP extensions much easier than before, allowing developers to create amazing performance enhancements through custom PHP extensions. Zephir is still a young project. However, normal project growth pains such as fragmented documentation and lack of distribution packages needn’t prevent use of the awesome new tool.

Zephir Installation Overview

There are two top level tasks when installing Zephir. They must be done in the following order.

  1. Install Zephir Parser first! – The Zephir parser is a PHP extension that needs to be installed on your development machines php-cli. You must install the extension before you can install the Zephir compiler. The Zephir compiler documentation does not say this explicitly enough. There is a dependency by the Zephir compiler that requires the Zephir Parser PHP extension to be installed and working for the compiler to install and run properly. Like any extension, it is basically a matter of dropping the zephir_parser.so into the right directory, and putting an `extension=zephir_parser` directive into the right .ini file. Now, here’s the tricky part.

    When you run the install script that compiles the Zephir Parser extension and should place the .so file in the right place, but you really benefit from running the installation script with the correct flags pointing to the target version of PHP that you’ll be using for development. So, run the installation script with the following flags adapted for your version of PHP and operating system:

    sudo ./install –phpize /usr/bin/phpize7.2 –php-config /usr/bin/php-config7.2

    That’s the trick to getting Zephir Parser to compile on Ubuntu 16.04 against PHP 7.2 as of February 2018. I suspect it may be blocking potential users on other distributions as well. It is buried in the “Advanced Compilation” section of the readme, but the flags for the install script currently have no documentation and the need in some instances to compile with the –phpize and –php-config flags needs to be more prominently stated.

  2. Install the Zephir compiler only after you’ve installed the PHP extension on the php-cli command line tool. Use `php -i | grep zephir` and look for evidence that the Zephir Parser extension is installed. Only then should you attempt to install the compiler.

That’s the high level process. Most everything else is obvious from the documentation. Hopefully, this overview helps at least one other PHP developer get up and running with Zephir toolkit and start learning the Zephir language, which is the fun part.

Zephir and PHP 7.2 Rule!

I’ve been working with PHP since 2000 and I still enjoy using it. And now, with the release of PHP 7.2, which has all the good parts of PHP 7, including increased performance and a better type hinting system, PHP 7.2 fixes a number of bugs that impact security, such as a broken random number generator that didn’t produce cryptographically useful random values. I think if developers begin utilizing the extension system more often through Zephir, the combination of improvements in PHP 7 and an easier path to developing language extensions, will allow the PHP community to build faster, more robust applications and the PHP platform may regain some of its respectability. Zephir and PHP 7.2 and bring new life to a development platform that deserves more respect than becoming yet another legacy language.

How to Hash a Long Password with bcrypt

Hashing is an important competent in securing any application that users will log into, such as a Web application or an API for an an app, particularly when authentication is done using passwords, and bcrypt is currently a standard, acceptable way to create a secure hash. bcrypt’s acceptability may change in the future, but what won’t change is the reality that no sane person stores user password values in a file or database. Let me say that again. No sane person stores user password values in a file or database. Instead, the current best practice is to store a one way cryptographic hash based on the password and later, when the user logs in, hash the password that the user supplies with the one previously stored on the system.

Beyond simple hashing of a user supplied password, there are other use cases for wanting to securely hash the entirety of an arbitrarily long string using bcrypt such as constructing authentication tokens. One may also wish to salt the password before passing it into bcrypt for an even higher level of entropy.

One algorithm I use to get around the character limit is to use an intermediate hash. So, instead of hashing the password directly, I use a weaker hashing mechanism that produces a shorter string within the length limitation of bcrypt and then hash that intermediate string using bcrypt. The intermediate hash needn’t be strong hash, like bcrypt, though if you use a highly randomized and long salt you will maximize the effectiveness of this approach. My current intermediate hash is sha512.

A Recipe for bcrypt Hashing Passwords Using and Intermediate Hash

The steps I use for hashing ginormous strings with bcrypt, such as heavily salted passwords, are as follows:

  1. 1. Generate a long, cryptographically secure salt using a source such as /dev/urandom. (Do this per user. You can also append an application-wide salt as well if you like.) You will need to store this salt if you plan to do password validation in the future.
  2. Concatenate the salt to the user supplied password.
  3. Feed the concatenated value to sha512 and capture the result, which will be a 128 character string representing a hexidecimal number.
  4. To add entropy and make use of the entire intermediate salt, loop through the entire sha512 hash taking pairs of hex digits, converting the pair’s value to an ascii character and using each character to build a compacted intermediate hash that is 64 characters long where there are 256 possible characters.
  5. Feed the final compacted intermediate hash into bcrypt with your preferred parameters (such as cost) and capture the resulting hash.
  6. You now have a secure hash that utilizes the entirety of the value of your long password.

This is a simple solution to the problem of limited string length support for the bcrypt hashing algorithm, but there is room for improvement. For instance, using some simple stupid math tricks, one could create variations in the length of the hashed string, which up to a minimum viable length could be used to produce a larger number of possible hash-able values. To the extent that variation and unpredictability continue to make hashed values harder to guess, adding procedural complexity to this solution is really up to the creativity of the individual implementing it.

PHP Sample bcrypt Intermediate Hash Code Example

<?php
/**
 * A simple demonstration impementation in PHP.
 * This code snippet requires PHP version 7.2 or higher. Check
 * out PHP 7.2+'s improved encryption capabilities at
 * https://php.net
 */

// This is the string that needs a one way hash. A clever user
// could make it longer than bcrypt can handle.
$password = 'spicy pony head rules';
echo "The example password is: {$password}\n\n";

// Create hypothetical cryptorgraphically secure random values for
// an application secret and a user secret. Secrets should be quite
// large so they cannot be easily guessed or brute forced.

// The application has a secret stored in a private file. It should
// be randomly generated.
$applicationSecret = base64_encode(random_bytes(2048));
echo 'Application Secret: ' . substr($applicationSecret, 0, 48) . "...\n";

// The user secret is stored with other user data. Each user gets
// their own random secret.
$userSecret = base64_encode(random_bytes(2048));
echo 'User Secret: ' . substr($userSecret, 0, 48) . "...\n";

// If you concatenate these strings, they are too long for bcrypt to use the
// entire string.
$stringToHash = $password . $userSecret . $applicationSecret;
$len = strlen($stringToHash);

echo "The hashable string is {$len} characters long.\n\n";

// So, lets use sha512 as our intermediate hash since it gives us 128 bytes
// of hexidecimal values.
$intermediateHashedString = hash('sha512', $stringToHash);
$len = strlen($intermediateHashedString);
echo "The sha512 string is {$len} characters long. SHA512:\n{$intermediateHashedString}\n\n";

// Now, let's get full use of this string by taking pairs and using
// their numeric value to create a string of one byte characters.
// Effectively, we've created a base 256 modal number that fits
// the entire intermediate hash into bcrypt's input length constraint.
// (Don't output this to your terminal. It's likely full of control
// characters).
$base256HashedString = '';
for ($i = 0; $i < $len; $i += 2) {
    $base256HashedString .= chr(hexdec(substr($intermediateHashedString, $i, 2)));
}

$len = strlen($base256HashedString);
echo "The base 256 compacted sha512 string is {$len} characters long. \n\n";


// Finally, let's serve up our bcrypt hash.
$bcryptHash = password_hash($base256HashedString, PASSWORD_BCRYPT);
echo "Out final bcrypt hash is: {$bcryptHash}\n";

Listing the Current Branches of Multiple Git Repositories Under a Singe Parent Directory Using Bash

440px-strom_roka_borovica_velke_borove_03An interesting problem came up a work where I am maintaining a couple of legacy enterprise applications written in PHP. These applications were first developed over a decade ago and the version control was originally managed in SVN. Later they were moved to Git. I do not know the full reason why, though I have a theory, but both applications break down into multiple components, each of which is managed in a separate Git repository. This isn’t a huge issue, but it does create headaches when working on multiple tickets that impact multiple components. I want to make my changes for each ticket on a new Git branch for that ticket while leaving unmodified components on their master branch in Git. Unfortunately, there isn’t an easy solution to figure out what branch each Git repository is pointed at, a problem when there are perhaps ten separate repositories that all interact with each other.

Anyhow, yesterday I wrote a Bash script to address the issue, so I can now get a quick summary of what branch each repository is currently pointed at. Since writing the Bash script took a bit of research, trial and error (as Bash does), and was generally a bit of a pain in the ass (as Bash scripting is), I figured I’d publish my Bash script so that someone else out there might benefit from what otherwise would be a single use tool written for myself, and possibly a few interested team members working with these clusters of Git repos.


#!/bin/bash
app_path="${HOME}/path/to/application"

# Get all the repo paths for opsdb and store them in an array.
repo_paths=()
while IFS= read -r -d $'\0'; do
    repo_paths+=("$REPLY")
done < <(find ${app_path} -maxdepth 1 -mindepth 1 -type d -print0)

#echo ${#repo_paths[@]} # DEBUG Print the legth of the arrray.

# For each array, write the path and output the current branch.
for component_path in ${repo_paths[@]}; do
    echo -n "${component_path}: "
    git --git-dir=${component_path}/.git --work-tree=${component_path} rev-parse --abbrev-ref HEAD
done

SSHFS on OS X via Fuse and the Horrible Dot Underscore Turd Files

Once again, I have been issued a Mac to do development work, and as usual everything unixish on OS X almost works. To my delight, when faced with having to code against a Linux environment that I access via SSH, I found that since I last faced this situation, someone had made and is maintaining a port of Fuse for OS X with easy support for SSHFS. This discovery was a really great way to start the morning. I downloaded the installers, ran them, wrote my script to mount my remote machine to my filesystem, and had it working by the time I finished my first cup of coffee. Kudos to the team who maintain that port!

Of course, with the OS X filesystem in the mix, network attached filesystems never just work the way you want them to. In this case, the problem I ran into was the presence of files beginning with ._ the moment I created a new file in my repository and ran git status. I’ve seen these before. They pollute the files of most every filesystem ever touched remotely by a Mac, and they are totally invisible to the Mac user. They are used to store metadata for OS X, and are totally useless for any other computer. One Stack Overflow commenter deemed these dot underscore files “turd files”, which I consider an apt description. Apple calls the double underscore files “Apple Double” files. Whatever you want to call them, when being introduced into a client’s code repository on a Linux OS, these files beginning with ._ are turds and I want to prevent them from ever being created on my remote system.

It took some digging using a brand-name search engine, the name of which I will not disclose, but I was eventually directed back to the OSXFuse documentation, which, with a little reading, gave me a solution. (Cheers to working documentation!)

In the SSHFS connection string there are several options that can impact the creation of Apple Double (._) files. The two that I wound up using were noapplexattr and noappledouble. noapplexattr prevents OSX from even attempting to apply its extended attribute shenanigans to the SSHFS mounted operating system and noappledouble explicitly prevents the creation of any file beginning with a dot underscore as well as the also very annoying .DS_Store file. I’m not certain that both options were needed for my situation, but I used both and tested the result, and it seems to be working. Not almost working, just working.

My command for mounting my SSHFS looks like this (with sensitive bits redacted, of course):

sudo sshfs -o allow_other,defer_permissions,noapplexattr,noappledouble,IdentityFile=~/.ssh/id_rsa myusername@clientsserverhostname:/home/myusername /mnt/sshfs/mymountpoint

il_fullxfull-957561786_o31h
So long ._ files!

Server-Side Swift on Linux

“Can I write Web applications in Swift?” was a random question that popped into my head a few weeks ago after discovering Swift.org, a project that moves Swift programming language out of the domain specific role of replacing Objective C as the defacto language for building apps targeting Apple’s iOS and OS X platforms and into the Open Source ecosystem. The answer is, yes, thanks to the work being done by Perfect.

Read More »

Boycott San Francisco!

People love San Francisco. There’s the great food, the proximity to wine country, and a general lack of snow. San Francisco has a rich heritage of counter culture, dreamers, homosexual poets, hippies, hipsters, and yuppies. Yes, I said it: yuppies. Not that yuppies are inherently bad. Okay, yuppies are inherently bad, but having a few bad apples around helps remind you how good the good apples really are. Having a lot of bad apples, though, means that you just have a lot of bad apples.

I’ve lived in the Bay Area for almost 30 years, San Francisco for six, and never before have I seen things get so bad and desperate. In a time when most regional US economies are not producing enough jobs, the Bay Area seems to have done the opposite. We have overproduced jobs to the point where there is no more housing for all the people who are coming here to fill those jobs. It has gotten so bad that I have begun looking for jobs elsewhere, in places where it might snow even, because I cannot afford to live here anymore. But I can’t find jobs in tech, outside of the few core hubs, that pay a competitive salary or match my level of skills and seniority. What I do find on sites like Craigslist are companies posting in other regions ads that beg developers to COME TO SAN FRANCISCO.

Well, here’s a news flash for all those employers. About fifteen years ago broadband Internet became a thing for middle class consumers. You do not need to bring any more warm bodies into your offices to create synergy. Those warm coding bodies would be happy to take 80% of what you’re offering an on-site employee and quietly code at home in Denver or Minneapolis or damn near anywhere else you can think of. And they will be more efficient, because the last thing you ever want your software engineer doing when they’re hacking on your core product offering is worrying about the increasingly realistic possibility that they may be homeless while earning a low six-figure salary, because a low six figure salary will not pay for a modest apartment in San Francisco these days.

And if you’re a programmer on the outside of Babylon looking in, dreaming of taco trucks, cheap cabernet, and enlightened free spiritedness, think twice. The people who are making what elsewhere would be very generous salaries by moving here are subsidizing their lifestyles. They’re going into debt, living off of trust funds, or growing weed in their closet. No one here has a retirement fund.

Maybe this is where the VCs are. Maybe this is where the intellectual ferment of the digital age starts, but it doesn’t have to end at the edges of the Bay, or even at the city limits of (gasp) Oakland. You can take it home. You can tell that VC that you’re going to start your company in your home town and save tons of their capital in the process. You can create jobs in your community. And if all you want is a job, you can tell Robollox or whomever is poaching for talent in your community that you would love to join their development team and you’d be happy to fly in once a month for the requisite face time, but you would rather rot in Hades than to give all of what should be your discretionary income to a scum sucking absentee landlord in the most screwed up housing market in these United States of America. Please, take those companies jobs and money, but also, please don’t come here. There is nowhere to live here. There are people renting tents in their back yards for $1,000 a month. There are people renting those tents. You will pretend to have fun for a while even though you will have no time or money to enjoy this once fabulous city. Then you will get sick of all of it and leave. The actually fun people can’t afford to live here anymore. The ones left, the odd indigenous San Franciscan excepted, are the die hard core of uptight yuppie scum who enjoy the smell of their own farts and will pay infinite amounts of rent for the status of having the correct zip code.

Move here at your own peril. I recommend an all out boycott. San Francisco has ceased to be livable and no one can afford flowers to wear in their hair.

URL Rewriting via VirtualHost Directive for the Slim PHP Framework

I’ve been using the Slim framework on and off for the past couple of years. I find it to be a nice piece of glue to structure my PHP applications without having to buy into someone else’s entire toolkit. In fact, I use it because it lets me use whatever tools I want and works seamlessly with Composer.

One thing that I find problematic getting a Slim project up and running is the insistence of the documentation that the rewrite to the front controller on Apache be configured in an .htaccess file. This is fine for shared hosting environments, but on applications where I control the server configuration and want to harden my infrastructure by avoiding such pitfalls as putting my server configuration in my document root, I would rather not have to put AllowOverride All anywhere in my Apache configuration files.

So, without further discussion, here is a VirtualHost file that I used to get Slim up an running on Apache with no .htaccess file. This was done for Apache/2.4.7 running on Ubuntu 14.04LTS. It took me several hours to remember how to do this, so hopefully this saves someone else some time in the future.

<VirtualHost *:80>
        ServerName web01.dev.example.com

        ServerAdmin david.cloutman@example.com
        DocumentRoot /var/www/web01.dev.example.com/htdocs

        RewriteEngine Off

        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteRule ^ /index.php [QSA,L]

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined


        <Directory />
                RewriteEngine On
                RewriteCond %{REQUEST_FILENAME} -s [OR]
                RewriteCond %{REQUEST_FILENAME} -l [OR]
                RewriteCond %{REQUEST_FILENAME} -d
                RewriteRule ^.*$ - [NC,L]
                RewriteRule ^.*$ /index.php [NC,L]
        </Directory>
</VirtualHost>