Pragmatic 32: A Slice Of A Physical Machine

Welcome to Pragmatic. Pragmatic is a weekly discussion show contemplating the practical application of technology. Exploring the real world trade-offs, we look at how great ideas are transformed into products and services that can change our lives. Nothing is as simple as it seems. This episode is sponsored by ManyTricks, makers of helpful apps for the Mac. Visit manytricks.com/pragmatic for more information about their apps Butler, Kimo, Leech, Usher, Witch, Desktop Curtain, TimeSync, NameAngler and Moom. If you visit that URL you can use the code pragmatic25, that's pragmatic the word and then 25 the numbers, in the shopping cart to save 25% on any ManyTricks product. We'll talk more about them during the show. I'm your host John Chidjie and I'm joined today by my guest host, David Smith. How are you doing, David? Hello, and thank you for having me. It's great to be here. Cool. Thanks for coming on. Greatly appreciate it. Before we get stuck in real quickly, I just want to quickly thank Mick Travis for the lovely review on iTunes. Remember, if you're enjoying the show, please leave us a rating or even a review if you're feeling so inclined, again, would really appreciate it. So today, it's something that's been on my mind for a long time. And a lot of people who are listening to the show, have a blog, or have had a blog or are considering having a doing a blog. And I think that it's the best angle to sort of cover this from, from a point of view of most people's use cases and trying to keep in line with the show is one of those things is understanding if I'm going to build a site, where does my site belong? Do I have it on my own server? Do I have a server up in, and I hate calling it the cloud, but the cloud out there on the internet? And if so, do I get a dedicated server? Do I do a virtual server? Or do I just, to hell with all that, go with shared hosting or even to hell with that and go with a fully hosted solution, something like, well, I don't know, like Blogger or Squarespace or whatever. And I was glad when you picked this topic because this is something that I know that you've had a lot of experience with. Yes. And though, yes, I've done some amount of it on the blogging side and I've done a whole lot more of it just in the general experience with trying to find the right level of abstraction to build your things on, whether that's a blog or whether that's a database or a web service or anything that's going to involve a computer that has a persistent connection to the internet. There's a lot of questions that you have to navigate and spend a lot of time thinking all that through. Yeah. And I think we were talking just briefly before the show about the quantity. How many servers are you currently administering? It's somewhere in the ballpark of probably between 20 and 30. It varies based on various things in terms of load. The nice thing about modern hosting is you can very easily spin up and spin down servers as you need to, but it's usually somewhere in that kind of ballpark between my various apps and services and my site and my podcast and all these different things. You just end up accumulating all these different servers for all these different needs and purposes. that I've... I don't really have any actual training in this, it's just the kind of thing that I've just learned over time. Okay, cool. Yeah, well, the funny thing is, you say that it's something that you sort of learned over time. The number of people that, I don't know, that I've talked to that have learned how to do this stuff and become very, very good at it are the ones that have learned that way. And the formal training side, yeah, this doesn't seem to be as common, Which is, that's just been my experience, but in any case. Yeah. Well, and I think there's an element of it that, and maybe we'll probably deal with this a little bit when we get into understanding the various levels of abstraction that you can be hosting your things on. But a lot of it's just going to come down to your level of comfort. And over time, just by my nature, what I end up inevitably doing is you start with something that's very high level. You start with something that's very, has, you know, it's like you're playing with, what is it, Quatro, the giant Legos, and then you work, after a while you get bored playing with those, and you decide you want the Duplos, like the next smallest or whatever it is, and you start wanting to do something that you can't do with a high-level solution, and so you end up having to keep working your way down and down and down, or you start challenging more and you start approaching and working on more and more interesting and challenging problems, and so you end up like, "Well, I have to do this thing because I'm running into some kind of constraint, you know, performance, capability, etc. And just so every time you work your way down, you just learn something new. And so I think that's just sort of my path anyway, it's just been over time, you just kind of have to keep working your way down. And at this point, I've gone all the way to the bottom, I suppose, or at least to the point where the things that the decisions that I'm worrying about are as low level as you could reasonably go beyond running my own data center or something like that, you I'm still not worrying about that side of things, but as far as you can go on an actual server, I've probably gone all the way down to the bottom. So it's just kind of the experience that you run into after doing this for a long time. So the next step is the David Smith hosting service with server farm and everything in the backyard. Gosh, I suppose. That's one thing that I... My suspicion is this is where I will stop, where I'm actually on fully dedicated machines on bare metal in a data center. I don't have to know physically where that is and physically the number of redundant power supplies that it's connected to and how much diesel is in the diesel tank outside of the data center and all those types of things. At least that, I'm happy to just abstract that off and pay someone else to worry about that. In terms of everything else on the actual from the machine up is fair game at this point. Cool. Awesome. Okay. Well, I'd like to sort of, it's interesting the way that you've described that. I'm actually planning to cover this sort of back to front. So, which is, we'll start at the, what seems like the simplest. And I guess the reason I want to cover it in the reverse order is because the way I see it is it's been an evolution of complex to simple. So, from the very beginning you had no choice, you had a physical machine. And as things have abstracted away to make things easier, that has given people more options. So, whereas now there's always been the possibility of being able to go down to the level of dealing with a physical machine and that level of abstraction, you don't have to worry about abstracting away things like a virtual server and so on. So I think it kind of makes sense to sort of start from that point and then, but oddly it sort of sounds back to front based on the way you just described it. So okay. It's the same thing. Yeah, it's the same thing. Okay, cool. So pure and simple, and this is going to sound like maybe even a ridiculous place to start, but I'm going to start here anyway. And that is server versus a computer. And this is one of the things that when I first learned about computers, I sort of thought to myself, "Oh, what's a server? Is that like a, is this like a special, special computer?" And the funny thing is that, it may or may not be obvious, but a server computer is designed purely for one purpose, and that is to serve data out to other computers. And as a result, it has a different set of design optimizations. So you've got a desktop computer, it's going to have expansion slots in it, it's going to have a, like if it's a mini tower, of course, I'm talking about PCs more so than Macs at this point, because Macs are like all-in-one machines. But the idea is you've got a keyboard, a mouse, and a monitor, and they sit separate to the machine usually. And, you know, that's a different set of optimization is it's got a relatively inexpensive power supply, it's got relatively inexpensive components, like it might have relatively cheap hard drives in it, you know, nothing particularly flash. But when you're dealing with a server, you want something that's serving, you know, dozens, hundreds, thousands of people, you want to have something that's got a lot of redundancy in it. So you want to have something that's got redundant power supplies, because power supplies statistically are the most likely component to fail. and you're going to want to have higher reliability hard drives because these hard drives are going to be spinning constantly. So you'll hear the term server grade, so like server grade hard drives, and they'll be designed to spin for 5-10 years. That's assuming because they're still using that if you're going to SSDs, which is starting to happen more often. Then that's a different discussion and we've talked about that in the previous episode. So the point is that when you're going to a server machine, it's going to cost you more because it's going to be designed to be much more reliable and much more solid. Although technically there's absolutely nothing stopping you from running server applications on a standard desktop computer. If you really wanted to, there's as far as I'm aware, nothing stopping you. So, I mean, apart from common sense, perhaps, but. Sure. And it's probably, I think that's something that I've noticed in my conversations was just with, just generally with people where they're trying to understand like how the internet works is I think it's an important thing to make sure that it's like there is no difference like practically between a computer that's a server and a computer that's a desktop computer in terms of the things that you know and understand about that are applied to that just as well. The things that are different is that a server is just the way it's set up and connected to the internet and connected to the world is just different because it's designed to be on all the time and designed to be connected all the time. It's designed to have a much longer lifespan and to have a different, like you said, just different design constraints on it. But you could just as easily, I mean, this is sort of like Mac Mini Colo or people like that where they just take Mac Minis and hook them up as servers. There's nothing different about that computer. It's just the way that it's being used that's different. Aren't they submerged in oil? No, I think that's somewhere else, but yes. Oh, sorry. I just remember that from an ad on 5x5, but anyway. the things that stick with you. But you're absolutely right. And this is one of those little things that I just thought, "You know what? Maybe it's silly. Maybe it's not. I'll just cover that just from the beginning." No, it's good. Just so it's clear. So, okay. Everyone thinks to themselves, "Hey, I'd love to run my own server. I don't know why, but let's just do it." Okay, sure. So, where do you start? And back in the beginning, in like the beginning, if you had a server serving out data, you would run it up on your own computer at home and you would connect into the internet. And sometimes that connection most of the time was not permanent. You would dial up, connect, and then your computer would be on the internet and then you could serve data from it. So the problem of course with that, if you then progress a little forward in time and say I've got a permanently on connection, the problem has become not so much the power of the computer, more so the upload bandwidth seems to be the limiting factor. But it's not just the upload bandwidth, it's also the hassle because you've got to make sure that if you're serving all of these people, you want to make sure that your computer is running all the time. Okay, even if it is a cheaper computer, at least have a UPS on it. So I'm sorry, an uninterruptible power supply. And I'm sure most people know what that is. But just in case that's just as a battery in a box, and it's got a power converter in it. And essentially, when the mains goes offline, the battery continues to power your computer for a period of time till the battery runs flat. And then when the power comes back on again it automatically switches back to the main supply and starts charging the battery ready for the next use. Of course you scale that up with a diesel generator you mentioned before but most people in their houses don't have diesel generators but we'll get to that later. Yes they definitely do on larger scales. So that's sort of like where you traditionally would start for some people anyway. Now I'm not even saying that that was the most common instance but that is certainly where you could start. There's nothing stopping you from doing that if you really want to do that. So you can have your own computer on your desk at home acting as a server. But as soon as it starts to get a decent amount of upload bandwidth, you're kind of screwed really. Because yeah, most home internet connections don't have a high upload bandwidth. And frankly, yeah, well, okay, mine certainly doesn't. Mine's horrible. It's like ADSL 1, 384 kilobit per second up. It's atrocious. It's beyond atrocious, but never mind that. So anyhow, okay. So the next question is, what do you do? How do you then take that to the next level? And the obvious way is to put that out somewhere else on the internet, somewhere that does have a massive, fast, wide bus for uploading data. In other words, so everyone else to download, because it's all perspective, right? So I'm an uploader, you're a downloader, all that sort of rubbish. So what they would do is they they would host it in their premises, but they would have much, much bigger, you know, connections to the air, like T1s, E1s, depending on what country you're in, you know, or different, there's a whole bunch of different terminologies. I don't want to go into all of that, not this episode anyway. And all of that extra bandwidth means that they can serve out a lot more people more so than you can. And that became essentially the winning model because it was more cost-effective, you would get a much better result, you get higher uptime. And the other beauty of it is, of course, you then wipe your hands off all of that drama of taking care of the physical hardware. Because the other issue of course at home is that, you don't have a temperature controlled environment in your house or a humidity controlled environment. And that's obviously not good for the longevity of your hardware. 'Cause you wanna have that in a temperature controlled environment 'cause that reduces the thermal cycling 'cause thermal cycling drives a wear out mechanism on the hardware. As things expand and contract at differential rates. And then you've got obviously the humidity as well. humidity will drive corrosion or lack of humidity will prevent some corrosion. So honestly, those two big failure mechanisms, you take that out of the equation by putting your computers in a temperature and humidity controlled environment. And a host who's hosting this on the internet, wherever it might be, they can take care of that for you at scale and you simply pay them a nominal fee for that privilege. So that level of abstraction is pretty much the minimum and it's pretty much the standard That's pretty much what everyone does. And I think that's more or less where you said you've sort of stopped with your-- unless you're going to set up your server from the backyard. Yeah, and because the reality is, at its simplest, that's what you-- you're going to need at least that. And everything we're going to talk about from here on is going to be built on top of that. Because at its core, it is just going to be a computer in a rack, and like they're rack mounted so they're easy to manage. And if you've ever seen pictures of a data center, that's all it is. There's just these cabinets and cabinets of computers that are built and designed to make things very efficient to operate and manage. That if a hard drive goes, it's very easy to know where that is and to know how to replace it. If one of your-- rather than just having one network connection, they'll have multiple redundant network connections with different providers in different places and all these types of things that they've just taken care of. So that at its core, the computer that you're trying to run has power and internet all the time in a very consistent, a managed way. And at its core, that's really all a data center's job is, is to provide power and internet to your computer and to keep it cool, I suppose. And beyond that, you're building everything else on top of that infrastructure that they're just then providing for you. - Absolutely. And it's something that you brought up there that I didn't have originally down to talk about, but just really quickly in case anyone is not aware of this. Server machines, I talked before about the refinement of their hardware, and you mentioned rack mount and ease of replacement of devices. So once you're in that server design market, what you wanna do is you wanna have redundant power supply. So as I mentioned, and the redundant power supplies themselves on the good servers are hot swappable. So you can literally unscrew them, pull them out of the back and put a new one in and slide it in without having to turn the machine off. because obviously that's downtime. You don't want to try to avoid downtime, right? - Yep. - Yeah, and the next most common thing to fail is of course your hard drive, your spinning hard drives usually. And yeah, so what they'll have on most services, they'll have, I think the current one is serial, I'm sorry, S, oh, serial attached storage, SAS, SCSI attached, oh, help me out here, Dave. SCSI attached storage, serial. - S, S-A-T-A, is that what you're talking about? No, no, no, no, no, no, no, it's SAS, SAS. Anyway, and this is what happens when I go off on a tangent without just brushing up. But the last server I set up a few years ago, we were using some SAS drives, 120 gig, and we had six of them in a RAID configuration. I think it was RAID 5, or was it RAID 10? I forget which, anyway. It'll be one of those two. It'll be one of those. It'll be RAID 5 or RAID 10, depending on what your characteristics you need are. Yeah. That's probably another show. Yeah. - Yeah, it's another show, exactly. So we've got to be careful. So yeah, the idea is of course, that if any one of those fails, that you can, again, hot swappable, you can literally remove it from the front panel and replace it and put a new one in and it will do all the cloning and mirroring and everything. There'll be a performance hit obviously while it does that, but essentially there is no downtime. It's all managed by the service firmware and it just does its job and ticks along nicely. And there'll be an indicator light on the front that shows you, like you said, this one's failed. so you know straight away which one to go to. 'Cause there's racks and racks of these things. And that's the next thing to mention is, what is rack mount? Well, during the Second World War, I think it was the Second World War, maybe the First World War, America decided to go with the 19-inch rack standard. And the 19-inch rack standard sort of became more or less an international thing, 'cause well, America was sort of, you know, involved in the war in different parts of the world. And a lot of people were using their gear, and for interoperability purposes, it was great to have a 19 inch rack and you could simply fill it up with whatever equipment you wanted and it simply just bolts onto the front using cage nuts and cage rails and so on and they call these slots on the front cage holes and so on little known fact about my career is that my first paid job after university was doing some mechanical design as an electrical engineer I was doing mechanical design but never mind that and did that for about six months. And that was designing 19 inch racks for switch mode power supplies and basically telecommunications company uninterruptible power supplies. These things are huge, like 30 feet long UPS systems, they're huge things. So yeah, anyway, that was interesting. So there you go, I know way more about cage nuts and cage rails than any human should, but there you have it. Anyway, okay, bit of a digression there, but that's okay. So, physical hardware. So, and you talked to this briefly is of course, they have more access to different bandwidth, different internet connections, and hence you have much more bandwidth available to you. And the way that the hosts that will charge and they'll charge either a flat monthly fee, usually irrespective of the bandwidth, others will charge based solely on the bandwidth that you use. Sometimes it's a mixture of both. Like there's a minimum flat fee. And then if you go way over on your bandwidth, they'll charge you additional for that. Obviously it varies from host to host and it depends on the content that you're hosting and all that stuff. So rather than get into a comparison of the different kinds of hosts out there, I encourage you to do your own research and there's a lot of options. So although maybe it's just a quick ask, do you have any preferences? I'm currently using DigitalOcean, by the way, for my site. If you've got 20 to 30, I'm imagining you've got that spread across a bunch of different ones, perhaps? Sure. I have my physical... I have a few dedicated hosts for FeedWrangler, my RSS service, where I needed just some very specialty performance. I use those at SoftLayer, which is a dedicated hosting environment. It's physical servers that are just running my stuff. And then I split the rest of my stuff up. I have a couple at DigitalOcean and I have a lot at Linode, which is probably the biggest provider of, like the majority of my servers are there. I just, I've been using them for years and they seem to work pretty well for me. - Yeah, I think Marco's using them also for Overcast at the moment. It's quite a lot, it's quite popular at the moment. So, cool, all right. Okay, so with the advent of multi-core CPUs, I think more than any other advent of technology in the computing space has driven large scale virtualization. And prior to that, one physical machine was one server and that was it, nice and simple. However, when Intel, I think it was Intel that really led the charge on this, when I started to put multiple CPUs in the one die, so that was with the Core Duo, which was kind of, anyway, another topic. But okay, the point is multi-core CPUs. All right, and I think actually they start out on the server line, if I remember correctly. Anyway, anyway, point is that once you go to multi-core CPUs, you can start virtualizing a lot easier. And virtualizing software from VMware, for example, there's all sorts of different ones, but I know VMware is quite popular. And I don't just mean Fusion, I mean, it's a cross-platform. So VMware can virtualize on, I think it's, it also runs on, I think they've got a Linux version as well as a Windows version. So they've got different options for you. And what that allows you to do is you can dedicate one or more CPUs on your server to its own virtual server. And what it does essentially is it fires up as though it was a, its own standalone machine, even though physically it's sitting on one piece of hardware. So if you've got a server that's got 12 cores or 16 cores or something crazy or more even, then you can then split that up into multiple CPUs serving multiple machines, virtual machines as it were. And that sort of leads to the concept of a virtual private server, which instead of having a dedicated physical piece of hardware that you alone allocated, you can now essentially get a slice of a physical machine. And it sort of sounds a bit odd when you first hear that idea, but why that's an advantage is that that drives costs down because it means that now the hosts who previously had to pay for the physical space for a single piece of hardware, a single hardware server, can now get a dozen people on that same piece of hardware or more even. So it also gives other people options they can say, okay, well, you know, I want a single core on mine and a certain amount of RAM allocated to my virtual private server, or I'm going to pay a bit more, and I'm going to go and get two cores or four cores, and I'm going to have more RAM. And it all can be configured. Essentially, you know, on the fly, you don't you don't have to go and call them and say, Oh, I want to set this up, I want to do this and that. It's all usually pretty configurable. So for example, in the case of DigitalOcean, I can I can dial up, you know, an extra gig of RAM with more hard drive space. And I keep saying hard drive, they run SSDs, but anyway. And I can do that all through the web interface and it's all very straightforward. So it's really quite amazing to me anyway, how far it's come. And I remember listening to Marco recently talking about this, about how far it's come back from when he started Instapaper to where he is now with Overcast and all the server side options you've got now, it really is quite mind blowing. So I think virtual private servers are absolutely fantastic. And anyway. Yeah, and I mean, the biggest thing that I think that a virtual private server takes advantage of or is a recognition of is that most servers aren't running at full capacity all the time. Yes. That there is a tremendous amount-- most network services are very bursty in terms of their need. And most often, the simplest way to think about it is most often they're responding to people doing things and people aren't always doing the same thing all the time. If you're a photo service, people are going to be opening it up and downloading their pictures periodically. And that periodic nature, I think, lends itself to something like a virtual private server making a lot of sense where rather than having a server that's physically dedicated towards one task, you can take that server and split it up amongst a variety of different purposes and tasks and take advantage of the fact that more likely than not, they're actually not gonna be contending with each other very often. Sometimes that's gonna happen, and when that happens, the virtual software deals with that in a way to make sure that you have a baseline of, this is the performance that you can, at the very least, always get. But a lot of times what you'll end up with is you can actually get better performance than you would have otherwise, because if that base computer is actually fairly idle, you may actually be getting a faster experience than you're paying for because that computer is just sitting there. And so you can take advantage of that in a really cool way with a virtual server, but you have all the simplicity and advantages of, you know, you have no idea that there's someone else next to you, except for the occasional times of high contention. Typically, you have no idea that they're there, and so that makes your experience very simple, but it takes advantage of this and gets you a lot of power for a relatively low amount of money. Some of these machines are, that's crazy. You're talking about five, 10, $20 a month for a fairly capable computer attached to a very high bandwidth connection. That's pretty amazing that you can do that at that kind of a price. - Absolutely. Yeah, it blows my mind. It really does blow my mind that it's even possible. And it's all thanks to the idea of virtualizing because it's just driven that cost down. And I am glad you brought that up because you're absolutely right. A server utilization, it's so wasteful having a dedicated server. And if you're only using 10, 15% of it, then that's just, yeah. Whereas this way you can bring more out of it, which is again, it makes it more economical for everybody. But the first time I heard this idea And this is something that I come across. And at the moment I'm working on a project where we've got quad redundant servers. It's a oil and gas application, but anyway, it's long story. And the quad redundant servers are actually all virtualized. And all of the server instances these days that it's becoming more predominant whereby we're virtualizing instances of the SCADA software. And it's just the way it's going. But we have to be mindful of one problem. And that is that if the hardware dies, and of course, you've got redundant power supplies, you've got RAID configuration on your hard drives, and you've got highly reliable components, but failures still happen. So when you do lose a machine, if you've got 12 people using that machine, 12 virtual servers going, all 12 of those are gonna get wiped out in one go. Now, that sounds really bad because you're like, oh wow, this is even less, this is much less reliable than before. Why would you even risk that sort of thing? And I suppose from the point of view of you're a single consumer and you are going to host all 12 of your sites on one physical machine, that's probably not a good idea from a reliability point of view. You want to spread that over different machines, even if they're virtual. And honestly, that's probably obvious, But the thing is that if you look at any individual, assume for example, that all of your 12 instances are spread across different machines. If one went down, the other 11 people on your machine and they're in the same boat, independently, statistically, there's no impact to the reliability overall. So the argument is cumulative availability for the total number of users on that machine. So if you lose that machine, sure, 12 people are affected as opposed to one, that's bad, but from any one individual's point of view, if they've thought it through beforehand, there's no difference whatsoever. So that's one of those little funny little conundrums, you sort of think about it and it sort of makes sense, but the initial reaction is, oh, gee, that's really bad, but it's actually not. - And what I find in my own experience is the reality of the way the pricing works on a virtual server is such that I can very easily add redundancy in a cost-effective way, because they're so inexpensive in the first place. For almost everything that I do, there's almost always two-- at least two versions of that same thing running, and then they're being load balanced and shared between the two. And so in some ways, that's even better than if I had one computer that was twice as performant. That would likely seem more dangerous than having two half as capable computers sitting next to each other being shared between themselves because I'm isolating myself from at least some levels of failure. Obviously there's still all these problems that, you know, some guy outside cuts the internet to the data center. You know, if both your servers are in that data center, you're still a problem. And obviously you can distribute amongst other data centers and so on. But I find in general, it's a very reliable way to do this because you're spreading it it out over such a wide variety of actual physical places that, you know, individual failures become much less likely or much less have a smaller impact when they happen, maybe, even if it's not really affecting the likelihood. Yeah, absolutely. Absolutely right. And that's something I wasn't going to actually talk about this, but I think it bears just discussing is the issue of, with respect to redundancy/load balancing, I think that there's two ways of looking at the problem. There's surges in load caused by lots of people signing onto a service at any one time or you've got a popular article on your site or something like that and there's a big spike and you could bring on additional capacity to handle that so that you don't experience a downtime. The other way to think about it is you're doing it purely for redundancy purposes. So you've got a known background static level of well, it's never truly static, but you know, an assumed static level of load and what you want to do is then distribute that such that if one fails, the other one is capable of taking up the entire load. And obviously with each additional serve that you add, it gets diminished by a proportional amount. So if you've got two, then I would suggest you would provision each of them should be capable of handling just a touch under 50% of your expected load as a worst case scenario, such that if one fails, then the other one will not quite hit 100%. And if you had a third one, then it's just under 33%. If you had a fourth one, it's just under 25%. That's the way I'd probably approach it. But from your experience, how have you approached that? It's along those lines, yeah. What you want to be able to do, and this is just a practical reason too, is that you want to be able to take any one server down at any point. Like in an ideal world, you want to be able to take anything down and have it as small impact on your end user as possible. That's like the number one goal that I'm striving towards is to be able to pull one thing out and have it not affect anything. And so with web servers or something like that, you want to be able to take one down and have whatever the other ones are handle that level of load. And they may be straining to do that, potentially, but it happens rare enough. But it's an amazing thing on a practical level to be able to, if I need to fix something on a server, that I can just take it out of rotation. And even if it hasn't failed, but it's just, I can take it out of rotation, do some maintenance, and then put it back into rotation. And no one was any the wiser that it just, that that just happened. That's a tremendously powerful thing to run a practical level to be able to do, which lets you prevent a lot of failures down the road in terms of being able to do that kind of preventative maintenance ahead of time. - Cool. Alrighty. So, next thing I just wanted to quickly talk about is now that we've said how awesome running your own server is, virtual private servers and so on, just want to mention how running a server can be a pain in the neck. (laughing) So these are wonderful. Oh, by the way, they're a pain in the neck. So, 'cause you need to, yeah, 'cause you need to worry about critical patches vulnerabilities, updates, you've got to check to ensure that the servers hasn't crashed, and might need a reboot or something. You've got to keep an eye on it. It's kind of like a child a little bit. You've got to keep an eye on it at all times. Otherwise, it's going to get into mischief. Well, yes, generally, that's definitely the case with my kids. That's a hassle. A lot of people either don't want to know about that, "Well, I don't know how to deal with that." So the idea along the way, before virtual private servers were a thing, back when everything was still physical servers was, "Well, why don't we do a shared hosting kind of an idea?" And the shared hosting idea could be done on a virtual private server technically, I suppose, but, you know, far more typically on a physical machine. At least that's the way they started out anyway. And user accounts are created on that same machine. So each user gets a slice of the hard drive space. Everyone shares a bit of the CPU time. And essentially, you know, obviously this impacts on the performance of the machine, but you know, it's not a dedicated, it's not your own playground like a virtual private server. You could load whatever software you wanted. This is a fixed set of software that they give you access to and they restrict what you're allowed to do. But honestly, if you've got a lower traffic site, that's generally not a problem. You don't need that extra performance. You don't need that extra configurability. You can just go with something sort of off the shelf and a hosted plan would do, you know, a hosted setup would probably do you just fine. So, uh, bottom line though is though, yeah. And I guess the same is true of any of these virtual private servers or shared machines is that if you've got one high traffic site on all the other sites on that machine will suffer. But again, you know, although that's less of an issue with a virtual server, it's still a concern because everything's going through the same hard drive essentially or SSD, whatever. So the thing with shared hosting though, is that, you know, it's the cheapest way you could possibly put something up there. So managing a shared hosting account is meant to be easy. It's meant to be straightforward for people that don't know Linux, don't want to know anything about Linux, or, you know, or whatever else or how to maintain a server, it's designed specifically for you. And the interface that seems to be most popular is something called c panel, which is like the letter C and then the word panel, all one word. And I had a little bit of a dig into this. I was curious actually about how old cPanel was, because it looks like something out of the nineties and that's actually because it is out of the nineties. So it was about 1996. Apparently it was created for a web host that's long gone, but it's sort of been adopted by the industry at large and everywhere seems to use this thing. So I've used Namecheap, Bluehost. There's a few other ones along the way, scattered ones I've used. They've all used cPanel. Now that is not an exhaustive analysis, obviously. And I'm sure there's others out there. In fact, that's a good question. Have you come across other ones other than cPanel? I think I've seen probably things that are forks of it. There's a lot of people who will take something like that and then customize it to their own needs and desires. But yeah, fundamentally, it's a very similar thing. Yeah. And it does, it does its job very well, which is, you know, takes a lot of the command line work out of it. It takes a lot of that specialized knowledge out of it. You still have to learn something. Obviously, if you're going to go to a shared host and you're going to set up a website, you still need to learn cPanel and what it does and how it works. It's a lot easier to learn, a lot more self-explanatory, at least I think so anyway, than diving in with CentOS, for example. >> Sure. I mean, what they're doing is in a shared environment, someone else is the responsible individual for that machine. It's no longer you. And in technical terms, like you're not the root administrator of that computer, more likely than not. And so you are given a very limited focused, secure, thought out set of things and commands that you can run and things that you can do on it. And so, the odds of you horribly doing something completely wrong are much reduced as a result. There's still obviously things you can do that would be unwise, but it's not quite the same as when you're... It's your computer that you've set up from top to bottom, that if you forget to do something or you do something wrong, that you can have horrible things go wrong, that you accidentally just deleted the entire hard drive or whatever. All these things that you can do, whereas most of these interfaces, it's like, you can do a lot less, but if you're just trying to take some files and put them on the internet and let other people view them, it works just as well for something like that. Absolutely right. And because it's cheap, it's very popular. So a lot of people will start there if they're messing with their own hosted solution, self-hosted solution. So before we go on any further and start talking about some of the different CMSs and advantages and disadvantages, I just want to talk about our sponsor for this episode, ManyTricks. ManyTricks is a great software development company whose apps do, well, you guessed it, many tricks, like the name suggests. Their apps include Butler, Kimo, Leech, Usher, Witch, Desktop, Curtain, TimeSync, NameAngler, and Moom. There's so much to talk about for each of those apps, what we're going to do is just focus on a different one each week, and this week, we're going to talk about Moom. That's M-O-O-M. So here's the thing. If you're sick to death of rearranging windows to make space to get your work done, MOOM can save you a ton of time. It makes it easy to move your window to the exact position on the screen that you want. Top half, bottom half, bottom left corner, fractions of the screen even. You could lock them to the top, the left, the right, whatever you want. Even better than that, it lets you, say, have, let's say you've got four or five windows open at the same time. I know I do when I'm at work, so I'll have Outlook open, I'll have my day, I'll have a calendar, I'll have a whole bunch of different things open, and it's always the same set of windows. What you can do is you can save and recall those window layouts quickly and easily using MOOM. Not only that, MOOM can automatically rearrange and resize your open windows when you add or remove a display. So let's say you've got a laptop, you're going to and from work, you plug it into your display when you get to work, it can automatically rearrange and resize the windows you have open when you plug in the new the external display. It's like magic, it really is very very cool. So the thing is there's so many little more you know some more little tweaks in Moom, it just it's better if you just go and have a look at it. So I had a play with Moom and I'll confess for the first time just in the last week and a bit and I'm really impressed. If you haven't tried it, download the trial and have a play with Moom yourself and see what you think. You can download a free trial of Moom from ManyTricks.com/moom and try it out. It's available from that page or through the Mac App Store for $10 USD. However, if you visit that URL before the 17th of August, you can take advantage of a special discount off of their very helpful apps exclusively for Pragmatic listeners. Simply use Pragmatic25 in the the discount code box in the shopping cart to receive 25% off. This offer is only available to Pragmatic listeners for a limited time. So take advantage of it while you can. Thank you. Very big thank you to ManyTricks for sponsoring Pragmatic. So now we've talked about shared hosting. I think it's time we sort of had a go at content management systems because I think that this builds on top of everything on the hardware and the physical, "Okay, here's a server, here's a host, here's why you would use it," and so on. The thing is that there's a lot of choice out there now. It's come a long way from just being a bunch of static files written purely in HTML, which is where I started back in '90. I think the first web page I did was '95, '94, '95, something like that. It was terrible. It was beyond terrible. And I actually still have the files because I'm a digital hoarder. I never throw anything away. Because I mean, it doesn't take up much space. I'll just put it on the hard drive and then I'll just burn that to a Blu-ray disc. And I think, yeah, it got it was it's gone from floppy to CD to DVD to Blu-ray. So it's been backed up forever. Memories. Anyway, so. I there are so many out there, it's so hard to know where to start. So what I'm going to do is I'm going to cheat and talk about my own personal experience, because frankly, well, that's just the quickest and easiest. And it gets the point across, I think. So the big option, the elephant in the room of self-hosted as well as with the option of fully hosted is WordPress. So have you played much with WordPress? Yeah, I mean, I hosted a variety of my websites on it for many years. Yeah. I've since moved on beyond it. But yeah, it's by far, it's probably the de facto standard for setting up a blog or blog-like thing on the internet and configuring that something where you're going to have some type of episodic content that you create and it publishes and manages the pages for, etc. It's built on PHP, I think. It's just a very simple... It's very simple but very capable is probably the right way to think about something like WordPress. It's a fairly general purpose tool but it does that general purpose very well. Absolutely and your story is almost exactly the same as mine and that's where I started as well. I started with WordPress. The thing is WordPress' predecessor was actually called B2 Cafe Log. Some people just call it B2 or Cafe Log. It was written by Mikkel Valdrigi and that was around about 2001-ish and as you say it was based on PHP and MySQL. It was forked by a guy called Matt Mullenweg and Mike Little in 2003 and that's when it got the name WordPress. So currently, the current version I believe is 3.9 at time of recording and it requires PHP 5.2.4 or greater and MySQL version 5.0 or greater. So those are the numbers there if you're interested. But anyway, it is open source, but you can host it at wordpress.com or at any number of compatible shared hosts. And believe me, there's a lot of them. You can install it on your own server, your own virtual private server, whatever you like. So wordpress.com has a free tier, But if you pay $99 a year, you get more storage, custom domain, more hard drive space, no ads and so on and so forth. This is not a sales pitch and no, they are not a sponsor of the show. But just for the sake of completeness, I mentioned that. There's also a business tier, it's got additional themes and so on. The thing I love about WordPress is the fact that there's a huge number of plugins, themes, lots and lots of people use it. It's, it's, I think, I'd love to know the numbers. I couldn't find the numbers, but surely it's got to be the most popular one in the world of its type. Probably. Probably, yeah. I don't have the numbers to back that up, but in any case, it's very, very popular. And what that leads to is that leads to there's a lot of support for it. So if you've got a question about anything to do with WordPress, you can Google it and you'll pretty much get an answer. In fact, you'll probably get a dozen or two dozen answers, and some of which might be conflicting. least the truth will be in one of them probably. Yeah. So, you know, so that's great. Whereas if it's a more obscure platform, you're kind of stuck figuring it out for yourself for the most part, which, you know, is something that I faced a little bit when I've been playing with Statamik, but that's okay. Generally, it was my fault. When I was doing something wrong, I just didn't realize I was doing something wrong. But anyway, so okay, the thing that attracts a lot of people to WordPress is it's got a nice little web based user interface front end. And it It also has iOS and Android native apps as well, so that you can modify posts and so on and tweak different things when you're out and about. And that's great because it means that the most common platforms being web browser and the two most popular smartphone platforms you're catered for, which is really, really great. And it all just plugs into wherever you're hosted, whatever website you're hosting it on, wherever that may be. So WordPress is a dynamic content management system. So I'll talk about CMS's a lot, you know, CMS content management system in case I haven't already told you what that acronym was in case you didn't know that. But anyway, it's a dynamic CMS. So that is when you request a specific URL, the PHP will construct a page dynamically and it pulls data from the themes, from MySQL, from the actual, the text of the individual entries in MySQL. And it then presents a finished set of HTML, CSS, JavaScript files to the browser for display essentially. They're dynamically generated. So that means that every page load the server's doing a certain amount of processing as well as serving the final files to the browser. You may say to yourself, "Well, okay, what's the big deal?" I mean, the big deal is that at a certain point, that can become an issue if you've got a lot of requests coming in. So that can slow you down sometimes in some cases. Now, I use WordPress for about three years, and I even set up a company website using it. And honestly, it is pretty solid, but at the same time, it can be annoying to customize it because I've customized a few different things that then interfered with certain other plugins. And if the whole point of WordPress is to make it easy, then customizing is perhaps not the point. Customizing up to a certain point is okay, but if you're starting to get in there and you're tweaking PHP and different things and even tweaking CSS, I found plugins would conflict sometimes with changes I made. Have you seen that scratching your head saying, "Hang on a minute, I updated that. Why isn't that being reflected on the site? I don't get it." It turned out to be a conflict with a plugin. You end up getting to this point where it's a great place to start but once you want to optimize it and if your blog becomes more popular, then at what point do you say, "Okay, well I've sort of outgrown WordPress. So anyway, okay so there are ways you can optimize and I found a great, well there's a lot of great articles, but I found one I've linked to in the show notes called the WordPress Optimization Bible and it gives you a whole bunch of, not a religious document, it gives you a whole bunch of ideas as to how you can essentially, what's the word, optimize. - Optimize, geez. That's like that thing in English where they say, describe the word optimize without using optimize in your description of the word optimize. - Yeah. - Yeah. Anyway. - So you can tune your performance. - Thank you. Yes, that's exactly what I meant. So there you go. And caching is the obvious one. And there's WordPress, the most popular plugin historically is WordPress Super Cache. There's actually a newer one out I've heard that's a lot better. and or it's making inroads and becoming more popular and it pre-generates the pages for you. So it essentially becomes a, it uses the dynamic only when you first generate the page and after that it's fully static. So that reduces the amount of server load every time you get a request. There's also other things you can do, like for example, turning off all the backups. 'Cause one of the things is every time you hit save draft on an entry in WordPress, it saves an entry in MySQL in the database and the database becomes this massive, huge, bloated thing. You use it for a couple of years and you put out a couple of posts a day, that thing gets enormous. And it's full of drafts. It's like 95% drafts, depending upon if you've got an itchy left mouse button and you're always hitting save. Force of habit for us, you know, geeks, right? - Oh yeah, sure. - Never trust autosave ever. - Never. - Anyway. Okay, so not too much else I want to say about WordPress specifically, but what I do want to talk about is on a shared host, they have this, almost all of them that I've, in fact, all of them that I've used, and I imagine almost all of them, have this thing called Softaculous. Have you come across that? - I don't think so. - Okay, well, Softaculous is essentially an auto-installer scripts for the most common platforms, including WordPress, but they've also got Joomla, Drupal, and those just come to mind. Some have mobile type, whatever. They've got lots and lots of them. And the idea is you don't have to upload anything with an FTP client or anything like that. You simply just double click on this thing and it guides you through, okay, what do you wanna call your site? What username are you gonna have for your entry? Okay, my entry, my name will be John Chigi. Okay, that's easy. And honestly, yeah, it generates it all for you, sets it all up for you. All you gotta do then is log into the website, the webpage front end and start working. So it really, they make it as easy as possible. And there's a link to Softaculous in the show notes and just if you're interested. So run a script and then go, no further setup required. That's the idea and it works pretty well. So I've used that multiple times and I've done it the harder way as well, but I create a directory and upload the files done the install that way, which is really not that much harder, but still, you know, it's a bridge that some people don't want to cross, I suppose. Anyway. Sure. It requires that you know a much, have a much deeper understanding of things. Yeah. It's not necessarily more difficult, but you have to actually know what's going on rather than just it being fine with, I push a button and then the website appears. Yeah. You don't need to actually know that, well, you're creating a file and then in that folder, putting a variety of different PHP files that are being run and based on which path you're accessing on the site, etc. You just don't have to know that. Exactly. And it's just a little icon at the bottom of your C panel. Softaculous installers, away you go. And you simply go there and select the one you want. And you don't have to worry about any of that other stuff. Which is, for a lot of people, is just what they want. So, anyway. Now I did mention Statomic and we'll talk a little bit briefly about that later but sometimes people don't even want to deal with signing up with a host. You know, like, so let's say, because that path would be I don't want to go on wordpress.com for whatever reason. Let's say I'm trying to save the maximum amount of money possible. The cheapest way to do it, let's say, and I want to have my own self-hosted solution so I have more control that I can choose to take my site wherever I want. So I don't want to be locked into WordPress.com. I could just go wherever I want. So I'm going to sign up with Namecheap or Bluehost or some that I've used or wherever. And I'm going to use their shared hosting and I'm going to install WordPress and I'm going to do what I want to do my way. Well, some people don't want to deal with that and they're perfectly happy to let go of another layer of control and say, "You know what? I'm just going to go to one provider that does everything." And that could just be WordPress.com. However, there's other ones like Blogger, Tumblr and Squarespace, we'll just talk about in just a second. So those sites are designed such that you log in, you create your site and away you go. You don't have anything else to set up, except to create content. And that perhaps is the most abstracted and ideal way for as an entry level for people that are trying to get into producing content for the web. So you don't even have to think about the machines or virtual anythings, which is great. Well, it's great. It's great. Great place to start. And a lot of people still do it even after they've been doing it for years, cause it just takes that complexity and stress out of it. So, okay, let's just talk about blogger then quickly. So, uh, it was mid 1999. So not quite in the two thousands yet. And, uh, Pyra Labs created this thing that they called blogger, but it only really became popular in 2003. Well, I'd never heard of it before then. And I think it only really became popular in 2003 when Google bought it. It wasn't technically my first website, but it was the first blog that I'd done in recent times. And that was for me in 2008, was using Blogger. And I found that it was very simplistic. But yeah, okay, look, it's very simple to use. Anyone with a Google account can create one. It's free. It's fully hosted. Yeah, but on the downside, it's difficult to customize it. And that's kind of, I guess, not really the point though, is that it's not designed to be customized. It's designed to be a simple starting point with a web interface and no hassle. Have you ever used Blogger? I feel like somewhere deep back in my past I have, but I think I'm enough of a control freak that I very quickly moved on to something that gave me a bit more control over what I was doing. I've definitely used things like it where it's that type of it's designed so that anybody can set up a blog. You just give it a name and you just start writing. It takes care of everything else. That's it. Hey, you're a fellow control freak. High five. There you go. Anyway, that's totally a good thing. Okay, so next one I just want to touch on briefly is Tumblr. That began in 2006, I think it was, with our mutual acquaintance Marco Armand and also David Karp, of course. Last year that was bought by Yahoo for 1.1 billion, I think it was. I think that's right. Something like that. Yeah. I mean, did Marco sell out too soon? That's the question on everybody's lips. But anyway, it's like Blogger, and I say it's like Blogger rather loosely, and Marco's probably screaming right now as he's listening to this, but it's kind of the next evolution. It has a lot more features and themes and following and reblogging other people's posts. creates more of a community feel to it than Blogger seems to. Whether or not that's, you know, that's just my sense anyway, is that I find Tumblr to just be a far more attractive, usable, nicer product than something like Blogger. Blogger is just like, you know, typical Google. It's like cut back to the bones, bare minimum, you know, this is what you need to do your thing. That's it. Not much from the way of flourish and that's okay. So anyway, but the great thing of course about both of those options is that they're free. They're not going to cost you a cent. Mind you, the site owns you or your content lives there. If that site goes down, then you're kind of screwed and you can't, you know, I think you can export information. Probably, yeah. I don't know for Tumblr. Most of these things have some type of give me all my stuff, but the question then of of course becomes, well, what do I do with all my stuff? Yeah, exactly. You have to work out a way of somehow translating that into somewhere else or finding something that can import that data into a usable format or who knows. But yes, you're giving up a lot of control, for lack of a better word, of your content, that you're very much at the whims of them and their services, their policies. If they change their policies, that could affect you in ways that you can't really do anything about. Yeah, exactly right. The point about being able to export data, remembering now actually with Blogger that the time, I don't know if that's changed, but certainly at the time years and years ago when I was leaving Blogger and then I went to WordPress, is I ended up just copying and pasting everything manually via the webpage because it was just easier. Sure. Which was great because I only had something like eight or nine articles that I'd done over a year. was less than a hobby. It was less than a part-time of a part-time hobby at that point. But honestly, if I had vast volumes of blog posts, that would have been agony. And when I went from WordPress to Statomic, it was easier, but I still had to write a bunch of scripts to clean things up. And even then, I had to go through and manually check all, I think it was up to 200 posts at that point. I had to go through and manually check each and every one. And every now and then in the back catalogue, I still occasionally, when I reference a site, will come across a straight underscore lying around. Oh sure. It's anytime you move from something like that. That's always going to be the case because you're not storing your blog, you're not storing your articles in a way that is designed to be transferable. designed to be a common language that everything else can speak. You're putting it into their system, and it's optimized for the way they do things and the way they store data. And so I think this is where we're about to head. A lot of people, what you end up wanting to do is to take your data instead and store it in some kind of more common format so that you're no longer tied into anybody specifically. So that exporting and importing is kind of trivial. That's probably, in some ways, the ideal goal. Yeah, exactly. So the next one to talk about just briefly, and then we're not going to talk too much more about CMSs because this is not meant to be the CMS comparison show, whereby you pick the best CMS ever made, because I'm pretty sure that's an argument that no one can ever win. So we're going to go with Squarespace. And no, they are not a sponsor of this particular episode, although they have sponsored the show in the past back when I was at Fiat Lux, but anyhow. So it's a lot more advanced and in many ways more flexible and more configurable, but it also costs. It's $8 a month, that's if you sign up for a year, and it has a nice drag and drop style of interface. But the great thing is you can also modify and tweak a lot of the underlying code. You can inject different bits and pieces and a lot of people, a lot of geeks use Squarespace and because, you know, well, they like it. And there's a lot of, I hate to call them hacks, but essentially they are in order to get certain functionality to to work. But the point is that there is a reasonable support base of people and solutions out there. It's one of those dig as deep as you dare kinds of solutions. It's got all the benefits of being a fully hosted, fully self-contained solution. but it also has a lot more, it's a lot more extensible. Is that the right word? You can dig deeper anyway and play with it more so than you can with Tumblr and Blogger, which you pretty much can't mess with beyond their theme appearances and so on. So, do you have a few sites on Squarespace, I think, is that right? No. I've never, I've created an account there to see how it works, but I've never actually hosted anything on it. I had a site there for about three months and I just decided to transition it and I was on WordPress at the time. I wanted to have a play with it to see if it was for me. Turned out that it wasn't for me. And I, you know, honestly, I actually think Squarespace is wonderful. But at the same time, I was sort of at a point where I was more of a control freak and I I'd recognized the fact that I wanted to have more control of what I was doing and Squarespace was more ahead. Okay, here's just real quickly. It had better uptime than my shared host did at the time. So it was more reliable, but it just didn't, it just, it just clipped my, clipped my wings a little bit too much. And had I dug deeper, I probably could have figured out how to do what I wanted to do, But I found out how to do it in WordPress, so I just stuck with WordPress. >> Well, and with any of these types of solutions, what you're doing is they're designed to very elegantly and completely solve a fairly specific problem domain or whatever it is. If you're trying to do what it's designed to do, it'll be amazing and fluid and great. As soon as you start to step off that path and you start trying to do things that it's not designed to do is often possible, but it becomes increasingly more difficult and it becomes increasingly less worthwhile because the whole point of something like that is that you don't have to worry about all these things. And so if you're hacking it up in some way that you're still worrying about it, it's usually end up being wiser to just take your step, step over onto something that is actually designed to have that flexibility in the first place. Absolutely right. Yeah. Couldn't agree more. So the last thing I want to talk about before we get into a bit of a go back over and review what we talked about is this phenomenon that I'm seeing from time to time about people that roll their own CMS. And I find it to be interesting because the vast majority of people who are doing this sort of thing that want to actually create content and put it out there. They're just not going to do their own. In the same way that you're not going to build your own server in your own temperature controlled, humidity controlled room in your house with multiple power supplies coming in from different power sources, a diesel generator out the back, multiple internet connections coming in, you are equally not going to go to that extent of rolling your own CMS because you've got WordPress, have got dozens, if not a hundred people working for them. I actually don't know the number, but I imagine they've got a reasonable number of people working from them. They've got a code base that goes back over a decade. They've got a large testing environment. They're doing development all the time. WordPress sites are being attempted to be hacked into by hundreds of people all around the world at the moment. And all of those attacks are forcing them to improve and strengthen what they've done. You're getting denial of service attacks that are essentially helping them to streamline the PHP and make it more efficient. Yeah, I just keep thinking to myself, why would you roll your own? Apart from the fact that it's like, well, gee, it's Sunday afternoon, I'm bored. Let's roll my own CMS. - Sure. - And the most recent example is, I just wanna mention briefly, is Casey Liss's Camel, and it's based on Node.js. And he rolled that recently himself. And of course he was on the show recently as well talking about military software, but it was, when he said he'd done it, I just sort of like shook my head. I'm like, what, you did what now? So anyway, I had a quick look at it and yeah, it's interesting. But I sort of say that with my tongue in cheek a little bit, but the truth is that John Gruber has been running daring fireball off a modified version of movable type. And, you know, once you start modifying things, is it the same as rolling your own? I don't know, but it takes you off of that upgrade pass. If I go and modify WordPress too much and hack into it, like this is inevitably what I ended up doing. It's not as bad as rolling my own from a point of view of testability and, you know, stability and reliability and maintainability and all those ability words. it's more about the fact that I've deviated from the standard setup such that if I then upgrade, how much of my stuff will break because they're going to upgrade the core functionality. So WordPress, you know, 3.10 comes out or 4.0 comes out and suddenly all of my tweaks and hacks don't work anymore. So then I'm stuck and I kind of get the feeling that either you're in it or you're not. If you're going to actually going to roll your own, you have complete control, complete ownership, but if you're going to start modifying an existing mainstream one, you're kind of sort of rolling it a little bit yourself because you're then locking yourself into the mods that you've made without perhaps intending to, if that makes sense. Yeah. So, honestly, I think that there's a balance and a lot of people do it, I think, just for, you know, and I hate this expression, but that's why I use it because I hate it, is geek cred. It's like, Hey, look at my geek cred, man. I rolled my own. It's like, well, okay, cool. That's great. Cool bananas. Love it. But, um, honestly, is it really going to be as solid as reliable as maintainable as something that you're going to get from WordPress, Datameg, Drupal, Joomla, you know, list goes on forever and literally does go on forever. There's that many of them, then I would suggest perhaps not. >>Steve: Yeah. I mean, where I've ended up settling myself is kind of in the middle there. So I use something called, it's a Ruby static site generator called Jekyll. >>Ashkahn: Yep, yep. >>Steve: Which, I mean, because ultimately what I ended up wanting to do, and my motivation for going this direction, is that it's sort of to the thing that I was alluding to earlier, I wanted my content that I'm creating to have longevity beyond just whatever platform I'm currently hosting it on. And so what I like about something like this is my blog, and every word I've ever written on it, is essentially just a folder of Markdown files. And that's all it is. And then I just have a bunch of scripts that take that and transform it into my website whenever I do an update. And the great thing about that is if I really wanted to shift from one system to another, if I wanted to roll my own, if I wanted to adapt to someone else's own rolled solution, almost-- it's a very transferable, non-- maybe proprietary is not quite the right word, but sort of. It's a very non-proprietary approach for storing that information. And so it gives you that tremendous flexibility and sort of peace of mind that in 10 years, if I want to still have that content available in whatever form, I'm not sure if it would be relevant, but if I did, I don't have to deal with deal with kind of the questions in the back of my mind. It's like, well, how would I get it out? How would I keep it moving forward with the state of the art? And so it's kind of exporting it to something that's very basic, very simple, that I can then just, you know, reliably move forward as technology advances. It's a very good point. And it's something that I was a guest on a Pocket Size podcast episode. I think it was 160 and we talked a lot about Markdown and the advantages and disadvantages of it and the data longevity and that flexibility is where it wins because yeah, Markdown is essentially, well, I say it's a standard, the original Markdown standard, of course, you know, been extended with multi markdown and there's a whole bunch of different flavors of markdown with different functionality. But essentially, it's a bunch of text files that don't take up much space, that essentially are cross platform and are very likely to be easy to import or with a bit of scripting are easy to import into whatever CMS of the future ends up being whatever it ends up being. And it's easy to transport between them. So you're using Jekyll, for example, I'm using Statomic. And Statomic is exactly the same thing. It's based on markdown files. So I could easily take my markdown files and build a site in Jekyll if I so desired. And it's that flexibility, or vice versa, you could come across to Statomic. But either way, and there's dozens and dozens of other examples. So yeah, absolutely. And I haven't really talked about this, about Statomic, is that I went Statomic. And what What Statomic is, is it's a static/dynamic site generator of sorts. What it does is, the first request, it generates a set of static files, and then from that point in time, you're working entirely from the cache. It has no database, however, it acts like a database. You build information into the markdown files, into the YAML headers, for example, and then you can reference it using a set of variable tags and different functions and it'll actually scan through them as the page is generated as though it was a database. It's kind of cool actually. It's a flat file CMS. It has very good performance just like Jekyll does and honestly, I quite like it. at the same time, it's required a lot of learning on my part. And this is kind of where I wanted to circle back to is, is the learning piece. So there's, there's, there's really a couple of ways of, uh, of looking at which is the right way to go for me and for my particular use case, because, you know, uh, the running gag seems to be only my use case matters. Well, obviously, you know, that's meant to be a joke because the truth is that everyone's use case is subtly different. So if you're going to do this sort of thing, what's the way to look at it? And I think there's really two ways to look at it. There's how much time do you actually have available to be messing around with the platform that you want to use? And if you don't have much time, so if you're, if you're time poor, as they say, then, you know, obviously a solution that is out of the box is far more appropriate to your needs. And the next, actually, you know what I just, I've written down too, but there's really a third one. So the second one is the popularity. And if you're running a site that gets a few hundred hits a day, not massive traffic, then you know, hey, you can get away with just running WordPress stock standard, no problem. However, if you start getting high volume traffic, you may start having issues depending upon where it's hosted and how it's set up. And of course, that requires more knowledge with caching and so on and so forth. Different CMSs will perform differently. So obviously that will also partly drive what you need to do. But the bottom, the consensus seems to be you want to go static because static or a static based system will give you the best performance. And certainly I'm imagining, well, confirm, deny, what do you think? Yeah, I mean the volume that you can serve from a static system is immense. Yeah. Certainly above what you'd ever have to worry about. Yeah, exactly. So, if it's a smaller site, you can get away with fully dynamically generated CMSs, but static is the preferred option if you want to go high volume. So, and then the third and final thing that should determine which way you choose to go is essentially your technical ability. And that is such an intangible thing because I'm a great believer that anyone can learn anything if they put their mind to it and if they're interested in learning it. And I never accept the fact that people are just not like technically inclined. I don't accept that. I believe that anyone can learn anything if they really want to. And frankly, I realized I just repeated myself as a method of trying to prove my point by repeating myself, which is actually not a valid way of proving a point. But anyhow, okay. Um, what was I saying? I saw I tracked myself, right? Yes. Yes. That anyone can learn anything. It's, it's certainly not a barrier. Not having, not knowing how to administer a Linux server is certainly no, is, is only superficially a good reason to not learn to administer a Linux server or whatever you'd apply that to. That's right. That it may be valid in the short term, but in the long term, if it's something that you actually would benefit from knowing and understanding how it works, then you certainly can. That's not something that should stand in the way of you considering something as an option. Yeah, absolutely. Especially with Google, the way that search engines have become so accurate. I'm thinking back to the days of the original Yahoo and Excite and, oh dear, all of those older search engines which were by comparison rubbish. Obviously, Yahoo has come a long way and it's a lot better now. Bing is passable, DuckDuckGo is excellent, Google is excellent in terms of its search capability. You can put a question out there, "How do I do blah in CentOS 6?" and you will get a a hundred responses. And that's exactly how I learned how to, cause I actually got CentOS 6 on my digital ocean droplet, my virtual private server that I run tech distortion on. And I've, I've learned how to, you know, set that up in the last, what, eight, nine months, just through Google searching. There was one point where Ben Alexander taught me a trick about how to how to get the tail end of a log file. That was very helpful. Thank you, Ben. And but apart from that, it's almost exclusively being Google. And frankly, yeah, I mean, you can learn this stuff if you want to. So. And I probably wouldn't need to say is I would encourage, I think anybody who's thinking about it, it is easier than it is perhaps. Then it seems superficially that at least in my experience, maybe it's just maybe that's easy for me to say being a nerdy tech guy, But is that a lot of these skills just build on themselves. And you only need to know as much as you need to know as you go. And so the way I started off being able to learn, for example, to administer a Linux server is there are a few basic things that I needed to do. Like how do I upload a file? OK, well, let's look up uploading a file. OK, I've done that. And then now that's just a tool in your tool belt. The next time you go in, like, OK, I need to upload a file and edit it. OK, well, I do what I did before. then I'll learn how to edit. And it builds on itself in a very iterative way that means that over time, you know, I've been doing this now for probably 10, 15 years, like I know now how to do a huge, huge array of things that if I had originally set out to try and learn at the beginning would have been very, very insurmountable. But it just builds up over time in a way that is much more approachable. Yeah, I agree with that absolutely and it comes down to that incremental learning. As you look at the ocean and I think the expression is trying to boil the ocean and it looks also hard. But what you do is you do one drop at a time and you chip away at it. And my experience has been I started out with Blogger which was completely hosted then I I moved to WordPress, which was self-hosted and I started out doing the Softacular script to install it and then I moved to FTP. Then I started tweaking the CSS because I wanted to customize the appearance. And so I learned more about CSS. Then I learned about JavaScript, then I learned about PHP. And then I'm like, "Oh, you know what? I should really do..." Because my traffic was getting more volume at that point. So it was like, "Oh yeah, I should go to a static system." And I was just getting annoyed with WordPress at that point. I had so many plugins and so much craft in there. It was frustrating me. Uh, I was breaking things when I was tweaking stuff. So anyway, so then I went to StataMic and a virtual private server, but that required more knowledge of PHP. I thought I knew a little bit of PHP. There's a lot to learn. And I thought, you know, I knew a lot about CSS, but I needed to learn more. And of course I had, okay, I'm going to install CentOS. Why? Well, someone said CentOS was good. So let's just have a go at CentOS. Fine. So I did that. And then of course, I'm learning about how to install, um, the latest version PHP using yum and all these different things I'd never known before and now I'm you know at a point where I'm running a virtual private server with statamix that's been heavily modified and yeah it's I look back at where I've come from and it's it's quite incredible really because I had none of that knowledge five years ago and none of it's all self-taught it pretty much it's not it's not something that I've gone and done a course for and and hence I I don't describe myself as a web developer or a web programmer or a web anything. I just describe myself as a hacker when it comes to this stuff anyway. Sure. But that's all you need to be. The reality is I think most people I know, that's where they are too. These things are accessible at least at some level enough that you can just do that and get away with it. The reality is, especially for most of these things that aren't necessarily your day job that are putting tremendous pressures on. If something goes wrong, you mess something up, it's like, "Okay, well, that was a learning experience," and then you just move on to the next try. Yeah, absolutely right. So with the three things that I mentioned, there's another thing that's come up time and again in this conversation, and that is control and being control freaks that you and I are both self-confessed control freaks. Maybe I should do a show called Control Freaks are Anonymous or something like that. I don't know. But never mind. Okay. So here's the thing. Ownership and control of the platform as well as the content tends to drive a lot of people to move away from fully hosted to self-hosted solutions with static files because that control gives us the sense, the feeling of control is what we desire, is what we seek. And we think of all of the advantages and, "Oh, it'd be great if I had this and I had that and then in future, then this would be easier." We talked about this. And honestly, if what you're doing is just for fun, or as a hobby, it's not a serious thing. And I look at tech distortion and I think to myself, well, you know, I would define that as a hobby. I really don't make, I make enough from the ads on there. I've got Fusion ads on there. That's it. I make enough from that to cover my hosting costs pretty much, you know. That's it, you know. So it's, in terms of cost, and this is the other thing, okay. Hobbies don't have to make money. Yes, I know they don't have to make money because it's a hobby. So you put money into a hobby and you have enjoyment from your hobby. I get that concept. at the same time, it's not a job insofar as I'm not drawing an income from this. Therefore, you're going to ask yourself, "Well, would it really matter if it went away?" And if it didn't matter, if it doesn't matter if it went away, then I would argue that the desire and the control for which you may seek is somewhat unnecessary because, well, it's just a hobby. And there's multiple ways of backing up what you've got. If you're thinking about it from a professional point of view, like with Feed Wrangler and Pod Wrangler, all of the stuff that you've created is a business. That's a completely different story. But for the majority of people that are listening to this, they're not thinking about the business point of view. They're thinking about it as a hobby point of view. And I just think it's important that people sort of get their head around the fact that The need for control often is a justification for itself, rather than actually a requirement. So, yeah. - And it's also probably good to say, it's a slightly orthogonal discussion, but so often, I know from my own experience, I will create things for me to do and fuss with as a way to avoid the task that I'm ostensibly trying to do that fuss to accomplish. So in the case of something like blogging, for example, it's like you can endlessly go around and around with tweaking your site in some ways to avoid having to write articles for it. Or you can worry about your mic setup and the software you're using to edit it, et cetera, and actually never ever record a podcast. Like at its core, that's, I think, something that's also just important to think about with a lot of these is if your goal is to create something, if your goal is to, you know, start a blog so you can put -- you have thoughts in your head that you want the world to know, in some ways it doesn't matter too much, at least from the beginning, to how you're getting that out there and making sure that if you're -- if you're getting -- you can get too, you know, down in the weeds about how you're going to do it and the different ways and all the different ins and outs, when really you could just, in some ways, you could just post test, you know, put a, you know, copy and paste out of a text file into something and publish it. Like it'd be fine. And so that's something that I know for myself that I always have to be on the lookout for is that I'm not being focused too much on taking over control of all these different parts of things and giving myself busy work to do because that allows me to feel like I'm doing something, but I'm not actually. Yeah. I feel like we just stepped into a back to work episode there, but you know what? That's awesome because I'm smiling, I'm nodding, I do exactly the same damn thing. You got to stop yourself sometimes and say, "Well, do I really, really, really need to tweak this little bit here? I should really just finish typing this and get this out." Anyway. All right. Anyway, all right. So, okay, so control, wrapping up on control. So if you self-host, you've got a little bit more control, but that control comes at a cost. You need to maintain it yourself. Need to make sure you've got the latest versions of let's say, WordPress updates and so on and so forth. And if you're using a lot of plugins and there's a lot of hassle updating that stuff. And honestly, yeah. So then you think to yourself, "Well, I'm going to do my own, I'm going to have a flat file CMS and it's going to be a lot easier. I'm going to actually have that control now. And then I'm going to say, "Well, no, actually, I don't trust these cloud host people. They're just not trustworthy or something. So I'm going to have my own physical server at home in a community controlled environment with a UPS and diesel generator and blah, blah, blah, blah, blah." And it's just companies that do that sort of thing, they have rock solid reliability or they're designed to, and they I have high internet availabilities, I have enormous budgets. You're never gonna beat that. At some point, you have to give over some aspect of your control. And everyone draws a line at a different point. For you and me, we've drawn our line at the virtual private server or the server hosted, our own server in the cloud. We've given up on shared hosts, I think, both of us. Is that right? - I think so. Yeah. I got too many things to keep track of, but I don't think anything that's important is hosted on a shared environment anymore. Okay. Hopefully that's true. And the person whose site is on there and you're like, I didn't mean your site in a bad way. Anyway, it's all good. Yeah. So we've all, we've gone away from that because that's where you and I, we sort of, that's where we've drawn our line and we're prepared to handle that abstraction and that level of control we surrender. That's fine. But the rest of it, yeah, the rest of it is, and using Jekyll's, I would suggest being Ruby-based. I mean, it has a lot in common with some of the aspects of Statomik, but I would say it's closer to the metal than Statomik is. - Sure. - And so you've probably gone just a little bit closer to the metal there, which is what I would, well, to be honest. - Yeah, well, it allows me to, I'm a Ruby programmer, so I can do anything I could want. If it's not doing something I want, I can never do anything because it's just a set of scripts at the end of the day. I always found the problem with Ruby is eventually it goes off the rails. Sorry, that is such a bad joke. I can't, I'm sorry. Okay, moving on. Actually, I think that's almost all I had. So, here's what I want to avoid at the end of this wrapping up is, this is what you should do. You know what? I just don't want to say that. And the reason I don't is because there are so many different people's ways of attacking this, ways of looking at this, that I don't think that's fair for me to say, well, this is the right thing that you should do. What I do think people should do is have a good hard think about what they're actually trying to accomplish and decide based on that. And be honest about your technical capability and see how far you can push it. Because some of this is fun if you want to get out there and do it. But if you're spending more time maintaining, developing and tweaking a site rather than actually producing content, I would suggest you're probably doing it wrong. And perhaps that's the parting piece of advice I'd like to leave. I think that's exactly right. I think it's one of the... we're spoiled for choice in the current environment where we can do anything at any level at some ways and you're always just balancing trade-offs. You're balancing your time, your money, the complexity of what it is, the portability of whatever it is, and there's a lot of trade-offs that you can make, but I think if you're honest about what it is you're trying to accomplish and what it is you're capable of or are desirous of learning about, then you can make the decision that makes the most sense for you for where in that stack you're going to be at any one time. Excellent. Very good. Well, if you want to talk more about this, you can reach me on Twitter @johnchidgey. That's J-O-H-N-C-H-I-D-G-E-Y. And check out my writing at techdistortion.com. If you'd like to send any feedback, please use the feedback form on the website. And that's where you'll also find the show notes for this episode under podcasts Pragmatic. You can follow Pragmatic Show on Twitter to see show announcements and other related stuff. I'd also like to thank my guest host, David Smith. And what is the best way for people to get in touch with you? You can find me on Twitter at underscore David Smith or at my website, developingperspective.com. And also your 15 minute long podcast, Developing Perspective. Yes, it is also hosted there as well. Absolute must listen. If you can't find 15 minutes to listen to that each week, there's something wrong with you. That's probably a bit mean, but no, strongly encourage, please listen to that. I'd also like to thank ManyTricks for sponsoring the show today. If you're looking for some Mac software that can do many tricks, remember to specifically visit this URL, manytricks.com/pragmatic for more information about their amazingly useful apps and use the discount code pragmatic25 for 25% off the total price of your order. And hurry, it is only for a limited time, so get in while you can. And thanks for listening everybody and thanks again, Dave. My pleasure. Thanks for having me. Anytime. [Music] [MUSIC PLAYING] (upbeat music) [MUSIC PLAYING] (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) [Music] [Music] One thing I just thought was a bit amusing that, so in the middle of that episode, I got a notification from Linode saying, "We have a hardware issue on one of your nodes." Oh, God. And then three minutes later, "We've resolved the issue and restarted your computer." And then I get the notification that everything's back up and running. Just made me laugh that, you know, while we're in the middle of talking about it, one of my machines had a brief hardware glitch that my host then immediately resolved. And everything's now back up and happy three minutes later. But it just made me laugh. >> That's brilliant. >> [LAUGH] >> Just while we're talking about it. >> Yeah, yeah. >> Linode's off doing their job. >> That's brilliant. [Chuckle]