How We Used Golang to Keep PHP From Dying
For the last 10 years, we have been developing software for companies of all sizes, from Fortune 500 businesses to small businesses with just 500 users. During that time, our engineering team primarily offered custom PHP web development services for backend purposes. It was 2 years ago when something clicked around our development that drastically changed not only the performance of our products but the scalability of them too . . . we introduced Golang (Go) into our development stack.
Almost immediately, we found that Golang allowed us to engineer bigger and up to 40X faster applications for our clients. We could leverage the power of Go to enhance our existing products written in PHP and take advantage of the benefits that both languages offered and replace the negatives.
We’ll explain how combining Golang and PHP development can solve real-world development challenges and become a new tool in your arsenal to solve some of the problems around the dying PHP model.
Let's Cover Your Everyday PHP Setup
Before answering how we can use Golang to resuscitate a dying PHP model, let’s start by describing your pretty standard PHP setup.
It’s pretty typical in most cases to run your application using a combination of a nginx web-server and a php-fpm server. Nginx serves the static files and forwards specific requests to php-fpm while php-fpm executes the PHP code. Less often, you can use Apache with a mod_php setup. Even though this method works slightly differently, it still uses similar principles.
For developers, it’s most interesting to understand how php-fpm executes the code of an application. When a request comes through, php-fpm initiates a PHP child process and the request details are provided as part of the process state (_GET, _POST, and _SERVER, etc). The state can’t change during the execution of the PHP script so the only way to get a new set of input data is to destroy the process and start over again.
An execution model like this one has a lot of benefits. You don’t have to worry that much about memory usage, all processes are perfectly isolated and if any of them do die, well, they will be automatically created without affecting the others. But at the same time, this opens up some of the downsides to this approach in PHP, when you trying to scale your application up.
How the Average PHP Setup Creates Extra Challenges and Inefficiencies
If you’re doing professional PHP development today, you already know the first step for starting a new project — pick your framework. Quickly, a framework provides libraries for dependency injection, ORM, translations, and templates. And, of course, all user input data can be conveniently located in a single object (Symfony/HttpFoundation or PSR-7). Frameworks are awesome!
But everything comes with a price tag. Any enterprise-level framework requires you to load at least a dozen files, construct multiple classes and parse a few configs only to process simple user request or query the database. The worst part is after each task is completed, you have to throw everything away. All the code you just initiated becomes useless now and will never handle another request again. Tell that to any developer who uses a language other than PHP and you’ll see the look of confusion overtake their face.
Over the years, PHP engineers have attempted to alleviate this problem by using clever lazy-loading techniques, micro-frameworks, well-optimized libraries, 2nd level cache etc. But at the end of the day, you still had to throw away your whole application and start over and over again.
Can PHP Survive More Than One Request With the Help of Golang?
It is possible to write PHP scripts with life cycles longer than a few minutes if not hours or days: cron jobs, CSV parsers, and queue consumers are all example. All these scripts follow the same process: retrieve the value, perform the job, wait for the next value to come. The code stays in memory the entire time, ultimately saving precious milliseconds as the myriad of interactions required to bootload a framework and an application take place.
Developing long-live scripts isn’t always easy. Any error completely kills the process, memory leaks are super annoying to diagnose and we can no longer use f5-debug.
The situation improved, however, with the introduction of PHP 7, which offered a solid garbage collector, made it easier to handle errors and prevented core extensions from leaking. Engineers still had to be careful about memory and state issues in their code (but is there any language where you don’t need to pay attention to this?). However, there are less unexpected issues you have to worry about.
Could it be possible to take the model for handling long-live PHP scripts and adapt it to more trivial tasks like handling HTTP requests and eliminating per request bootloading?
To start, we needed to implement a server application that could accept HTTP requests and then forward them to the PHP worker one by one without killing the worker each time.
We knew we could implement a web server using either pure PHP (PHP-PM) or write it with a C-extension (Swoole). While both approaches offered their own benefits, both left us unsatisfied and wanting something better.
We needed something more than just a web server. We needed something which could replace the negatives associated with heavy lifting in PHP but would still allow for easy customizations and extensions specific to each individual application. We needed an application server.
Could Golang help with creating an application server? We knew it could because it compiles applications into a single binary, it’s cross-platform, utilizes its very elegant concurrency model and a standard HTTP library, and on top of that, there were thousands of open source libraries and integrations we could use.
The Challenges of Making Two Programming Languages Work as One
First, we needed to define how two or more applications would talk to each other (inter-process communication).
One approach was to share memory between the PHP and Golang processes (similar to Apache mod_php) using an awesome library released by Alex Palaistras in the UK. However, it still had some limitations on how we could use it for our implementation.
We decided to use another more classical approach, where communication between processes was done using binary streams over sockets/pipes. We chose this approach because it had been a reliable method of communication used for decades and was well-optimized on the OS level.
To begin, we created a lightweight binary protocol to exchange data between processes and handle transmission errors. In its simplest form, this type of protocol is a netstring-like implementation with a fixed size package header (in our case, 17 bytes) that contains information about each packet type, it’s size and binary mask in order to verify the data integrity.
On the PHP side, we used the pack PHP function. For Golang, we used the encoding/binary library.
We went even further than just creating the protocol. We added the ability to invoke Golang net/rpc services directly from PHP. This functionality became very useful later in development as we could easily integrate Golang libraries into our PHP applications. You can see the results of this work in another open source product we released called Goridge.
Managing tasks across multiple PHP workers
Once communication was established, the next goal was to most efficiently pass jobs to PHP processes. With any incoming job, the application server had to select a free worker to perform the required task. If a worker/process failed or died, we would discard it and create a replacement for it. On the other hand, if the worker/process succeeded, we would return it back to the pool and make it available for next job.
In our implementation, we used a buffered channel to store the pool of active workers. An error watcher and state manager were put in place to pull any workers from the pool who unexpectedly died.
The end result was a working PHP server capable of handling any arbitrary binary job.
To make our application work as a web server, we had to pick a reliable PHP standard to represent any HTTP incoming request. For our specific needs, we simply converted the Golang net/HTTP request into the PSR-7 (https://www.php-fig.org/psr/psr-7/meta/) format which made it compatible with most PHP frameworks on the market.
Since PSR-7 is stated as immutable (some engineers may point out that it’s not technically immutable), it forces developers to write applications that do not treat a request as a global entity anymore by design. This plays perfectly with the idea of long-running PHP processes. Our final implementation, which we still hadn’t named, looked like this:
Introducing RoadRunner: A High-Performance PHP Application Server
Our initial test case was an API backend, which was periodically experiencing hard to predict bursts of requests many times higher than usual. While nginx was enough in most cases, it was fairly common to experience 502 error since we couldn’t balance the system fast enough prior to an expected increase in load.
In early 2018, we deployed the first PHP/Golang application server into the market to replace this setup. The effects were immediate and incredible. We not only managed to totally eliminate 502 errors but we eventually reduced the total number of servers by two-thirds which saved a ton of money on server costs and headache tablets for the engineers and product owner.
By the middle of 2018, we polished the approach, published it to GitHub under an MIT license and called it RoadRunner which described its incredible speed and efficiency.
How RoadRunner Can Benefit Your Development Stack
Introducing RoadRunner to our technical stack allowed us to use Middlewares for net/HTTP in order to enable JWT verification even before a request comes to PHP, handle WebSockets and globally aggregate stats into Prometheus. By using embedded RPC’s, we could expose any Golang library API to PHP without needing custom drivers. Most importantly, we can use our RoadRunner library to set up new servers that are different from HTTP. Examples include running AWS Lambda handlers in PHP, creating reliable queue consumers, or even adding GRPC to our applications.
To this date, with the help of the PHP and Golang development communities, we have improved stability, enhanced the performance of applications by up to 40X in some of our tests, polished debugging instruments, integrated it with the Symfony framework, and added support for HTTPS, HTTP/2, plugins, and PSR-17.
Conclusion
Some people still cling to decade-old feelings that PHP is a slow, clunky language you only use to write WordPress plugins. They might even say that PHP has a limit: once your application becomes big enough you have to switch to a more “mature” language and replace the years of PHP code.
To them, we say “think again.” We believe that the only limit PHP has is the limit you set. Even PHP mobile app development is possible when you intelligently design mobile applications on top of PHP backends. You can spend a lifetime jumping from one language to another trying to find a “perfect match” for your programming needs or you can start re-imagining the languages themselves as tools.
The ostensible shortcomings of one programming language like PHP may actually be the key to its success. By pairing it with another language like Go, you end up creating much more powerful end products than you would otherwise be able to with either language on their own.
After working with both Go and PHP for some time now, we can confidently say that we love both of them. We are not planning to sacrifice one due to the other, rather, we will keep looking for ways to derive the most efficiency from this type of dual-stack programming.
Spiral Scout is a leading software development company providing full stack development of custom products, ranging from small websites to large distributed systems, for clients in San Francisco and around the U.S. If you have a PHP or Golang related project or you suffer from an app written in PHP that can’t scale or is being weighed down by outdated code, please get in touch with our team at spiralscout.com.
RoadRunner was created by Anton “JD” Titov, CTO, Spiral Scout
