Pulsed Media: Inflation, Energy crisis and exchange rate changes are disruptive
Pulsed Media is not immune to market conditions, and the market conditions have dramatically changed over the past year. Energy prices started to heavily increase a year ago, but are now wildly changing week by week. Market pricing indicates a significant multiplication to electricity pricing with some people already receiving upto 10x the previous rates and 5x has become common place.
Our electricity price has already nearly doubled and we are afraid it could more than quadruple from that, to potentially 8x or more from the price we've enjoyed for the past 6 years in our current datacenter. Electricity price is, and has been one of the larger costs, but now it is already the largest. Some people are already receiving quadruple electricity price of what is our current already increased rate, and the best deals you can find right now are way more than quadruple of our original electricity cost.
There has also been discussions about potential power outages during the upcoming winter. How long and to what level, if any is yet to be seen. Currently Finland is on electricity deficit and has to rely on imported electricity, so a period without wind and extreme cold we could see power outages. Our datacenter is in Helsinki and next to a power plant, but even that will not guarantee we will not see power outages.
Electricity has always been a big part of our cost structure, and competitive electricity price and efficient cooling has meant that we have leaned very heavily on that advantage we've had in the past. Things have now changed with the energy crisis, and we are looking at very high electrical costs.
Our electricity is being bought in bulk, in combination with many other companies and for the long term, but even that manner is not immune to the market conditions -- typically this way of buying is a little bit more expensive than exchange market spot electrical rates -- but was stable and predictable. On the other hand, when electricity prices start going down our cost goes down slower than the overall market. Current public offers for electricity are more than 4x we used to pay already, with some companies charging even vastly more. Over 0.50€/kWh priced offers are increasingly common by the week.
Europe is not alone being affected by the energy crisis. The whole global economy is affected, albeit to a lesser degree than Europe.
Inflation, exchange rates and supply shortages
Inflation is very high as well, all operational expenses (OpEx) have increased over the past 2 years. Some costs have increased dramatically, not just energy costs. Official numbers hover around the 9% mark, Euribor 12month interest rate is going higher at a very rampant pace.
USD-EUR Exchange rate has dramatically changed this year, meaning that all hardware costs now roughly 15% more. Every service with USD basis in pricing costs that much more now. The good news is that those with USD as their billing currency are essentially now getting a 15% discount.
Finally we have supply shortages for some components, some things simply are not available at all, at any cost. In some cases we can find those parts for highly inflated prices, more than 2x the typical. Sometimes you hope to even find a "scalper" with the product you need in stock.
Our target market of low end, extremely good value for our customers has necessitated very precise calculations of all costs. A very precise pricing structure to be viable and profitable business for future expansion and to create ever better value proposition services year over year.
Some of the cost increase highlights over the past 2 years
* Electricity +~89% Estimate for Q4/22-Q1/23 is 400-800%
* Staffing +~50%
* Networking/Transit: +~23%
* Fuel costs: +~60 - 100% (fluctuates heavily)
* Buildings/Spaces: +~90%
* Accounting/Admin overhead: +~75%
* Hardware costs due to currency exchange rates: +~15%
We have managed to find some efficiencies as well which has boosted revenue without accompanying cost increases, but these things are not able to counteract even for all the inflation alone, nevermind energy crisis combined.
Price increases for services
Sadly, due to all these factors we have to increase pricing all around, for new and pre-existing customers. These will come in effect immediately or within a few a days. New pricing will be in effect on renewals and new orders, no mid payment term change. If you have an invoice already open for renewal that will be unaffected. New signup pricing takes a little bit longer to change as we have to re-do all the expense maths for all of the service ranges, these will happen by end of September, or with rough estimates before that.
Relative to revenue most expensive accounts are the smallest ones. The sub 5€ group of services represent about ~31% of services, but only ~10% of revenue, all the while being the source of about 80% of uncommon events. Seedboxes are highly dependant on the number of users per performance domain (ie. a server, or raid array) catering to this group actually increases production costs as well. On the other hand, the higher end service group is the complete opposite, therefore we will not increase pricing for that group as much but actually significantly less than actual increase in the operational costs (OpEx).
Therefore the pricing increases will be linearly adjusted based on service price 35-15% on 10-30€ services -- lower percentages on higher cost service, but also another 3.5€ for all sub 5.50€ accounts which are the most sensitive to OpEx changes.
The final pricing increases will be 35% to 15% scaling from 10€ to 30€ per month. Examples:
* 10€ current price, +35%, final price from 10€: 13.50€
* 20€ current price, +25%, final price from 20€: 25.00€
* 30€ or above current price, +15%, final price from 30€: 34.50€
This price increase will go to cover the increased operational costs.
Future plans and energy crisis mitigation options for the long term
Meanwhile we are starting a program to really start heavily lowering our electrical consumption. Several plans are already being executed, and we will get outside funding to achieve these goals if we have to. Pulsed Media is net debt free company currently and has been revenue funded until now.
Some of the plans are still aspirational, so we'll keep it brief. So keep this in mind, that some of these are currently aspirational and it takes time to move things to a new direction. Think it as a freight ship, even if you turn the rudder all the way now it takes a lot of time for the freight ship to even start turning around. Some of these plans has been brewing for more than a year now, so progress has already started on them.
First we got our second datacenter keys in August, once we build this out we are planning to install solar panels and the building owner is interested on buying our waste heat during the winter. This new datacenter is not very large neither, we are expecting roughly 400kw final capacity. Solar panels need some bureacracy, and we are not yet certain the scale we can build, but aspirationally we would like to get to somewhere around 600kw (+50% over the load) solar panel installation over the upcoming years, depending on the net metering we are able to negotiate. This site will have extreme economizer airflow volume capable of cooling more than 200kw @ 22 Delta-C by itself alone during hottest days of the summer. This will take significant time to develop, with multiple contractors involved and will be built out in phases. We expect very small production volume with compute only centric customers to begin during this winter.
Second, and easier to achieve. Extreme cost cutting by upgrading servers. Newer platforms consume less energy, while giving higher performance. No news there at all, but we are measuring in some cases even 110W per 1U server savings in electricity consumption. Secondary, we are looking into advanced storage setups such as effective NVMe caching right now and already have first servers up and running, this would allow us to have bigger performance domains // more users per performance domain. So far the test results have been _very_ promising for both upgrading to newer platforms and NVMe caching.
Newer platform will directly cut electrical consumption, but will still take years to pay off. NVMe/SSD caching would allow us to have more users per performance domain, while increasing performance. HDD performance, especially random performance, does not increase as the capacity increases so only way to increase number of users per HDD is by caching, or smaller performance domains (next item).
We have already started this process, but will likely take years to completely achieve.
Third, we are looking to partially virtualize and use bigger beefier servers split into multiple virtual machines for performance domain isolation. Therefore saving a lot in electrical costs per unit, while increasing performance. These were already on our internal development roadmap since earlier this year, and several very large storage servers are already in order with one of our server suppliers. This ties with the above hardware upgrades to further cut electricity expenses. These will first rollout to the Dragon-R tier of services (20Gbps RAID10).
Fourth, some legacy services -- really old services where storage capacity for end user is small we are looking to upgrade to SSD based service. These will account only for about few percent of our total electricity consumption however. Will also require hardware investments, therefore this is a minor item and will be done at the speed of convenience.
Fifth, we are going to beta test using regular consumer motherboards for regular servers like some very well known large brands in the dedicated server niche. We will begin with 2U-4U size for easier cooling. Currently only consumer motherboards we employ as servers is the ZEN MiniDedi lineup used for M1000 SSD seedboxes. They are absolutely brilliant for the task, but saddly currently out of stock everywhere. This has a big potential for long term performance increases and therefore decreased electricity consumption.
Sixth, UPS upgrade. We use Eaton PowerWare UPS systems currently. These consume a lot of electricity even when idling. Upgrading to modern transformless hybrid inverters could potentially pay themselves off just in the increased efficiency, let alone using lithium batteries instead of lead acid. Lead acid needs to be replaced every few years at great expense, and can be trusted only for approximately 50% of the rated capacity. Further, this should greatly increase reliability. This move would not make sense without huge increases in energy costs. This could also help if power outages happen, but sadly currently the USA to Finland sea freight is rather slow as well so there is no way we can get the lithium batteries imported by this winter. Typical normal UPS has battery packs for mere minutes of operation, and looses some of the extremely limited capacity each time they are drained. Earliest we can do this replacement looks like will be during next summer.
We might be forced to take the legacy type UPS units completely offline during upcoming winter if we get a really bad electricity rate, there is no sense running UPS when it cannot last over a blackout, and consumes huge amounts of energy. Our own experience and other datacenter owners all report that these units tend to cause more downtime than they save from, by a big margin. Hence, we were already considering changing them -- but that does require shutting down some devices for the duration it takes to replace the units.
Finally, we have to change the whole modus operandi based on increased OpEx and the risks associated with current crisis' -- at least until we get our new datacenter running at capacity and with sufficient solar arrays. This means we are going to target services a bit higher end, semi-dedicated and such as this shifts from OpEx centric to capital expenditure (CapEx) centric cost structure. Unfortunately, that means the days of sub-10€ services might be gone for a while for the most part. Or we find new ways to control resource consumption to acceptable level on the entry level packages, which is not a trivial task on seedboxes which are really hungry for those juicy IOPS! (IOPS = Input/Output oPerations per Second, storage performance).
Rest assured we are working hard on finding new efficiencies and keep providing ever better services year after year. Necessity is the mother of invention.
Not all gloom and doom
It's not all gloom and doom; We expect performance to increase on our services gradually as these plans get into motion. As part of the course for the solution to this energy and inflation crisis a lot of old time users will get service upgrades along with the hardware upgrades as well. This has been standard operating procedure, but never advertised or publicly mentioned before, to upgrade old time users after several years threshold to a notch higher grade service as the hardware and/or software (distro) gets updated. We really like our old time users, hence we do that along with the magnificent bonus storage quota. Bonus storage quota is about 23.44% of the actually allocated storage capacity for our users, on average. In other words, average user gets 123.44% of advertised storage capacity. Old time users can have multiple times the original storage capacity.
We also have new service launches coming this Q4/2022, hopefully all the effort needed to cut electricity consumption does not derail or further delay these plans.
You can discuss this at our Discord, link (valid for 7days): https://discord.gg/buTTbezZ
Saturday, September 3, 2022
