Pragmatic 87: Static vs Dynamic

Welcome to Pragmatic. Pragmatic is a discussion show contemplating the practical application of technology. By exploring the real-world trade-offs, we look at how great ideas are transformed into products and services that can change our lives. Nothing is as simple as it seems. Pragmatic is part of the Engineered Network. To support our shows including this one, head over to our Patreon page. And for other great shows, visit engineered.network today. I'm your host, John Chidji, and today I'm joined by Radek Pietruszewski. Did I get that right this time? Yes, you did. Hello! Alright. Hello! Thanks for coming back on the show. Thank you for inviting me again. - Yeah, no worries. I've been wanting to talk about a topic particularly recently because of my recent experiences and my, I guess you could call it my journey as a web developer, which I'm really not a web developer, but I kind of pretend to be sometimes in my spare time. And this is the argument, I think you call it, I'd like to say it's an age old argument, but it's not really that old as arguments go, this whole static versus dynamic thing. So I was, I kinda, yeah, it's been an interesting, I guess, journey. I don't know if you'd call it that for me. And I was hoping you can help me sort of flesh this one out a little bit. - All right, let's try and do that. - Alrighty, so way back in the beginning, which let's be honest was really only a few decades ago, like websites used to be pretty simple. a bunch of images, maybe an animated image, if you're really fancy, a bit of HTML, some HTML tags around the text, maybe a couple of files or two, maybe. I think we got PDFs back then and different things and you were done. But websites had to be developed by people that understood how to FTP onto a server and do all those modifications to the raw files and everything. And the first website I ever did was exactly that. And I actually had an animated Australian flag blowing in a virtual breeze, which thinking back was really tacky. But at the time I thought it was just amazing, but you know, it was, yeah, anyway. But it was fully static. And if you wanted people to email you, that was fine. You know, you just put your email address on the website. 'Cause I mean, nothing could go wrong if you did that, right? Just put your email address there, that's fine. Job done. And if you wanted to have some interaction, you know, you could do forms and basic forms that came along with no real encryption or anything. You just post all the details and away you go. What could go wrong with that either? And when I was actually first working on forms in particular, I was doing some work at Nortel and they had me doing some Perl scripts in the backend for some CGI bin forms back in '97. That was never externally facing, it was internal on Nortel's intranet. And that was all very interesting, I guess. slanty toothpick and all that other thing. And time moves on and the world moves on and websites move on. And I guess this is where we start to delve into the whole web 1.0 versus web 2.0 thing. So when you think of web 1.0 to 2.0, how do you think of that transition? Like which bucket is which, would you say? - That's a really good one. I recall Web 2.0 being a thing, a trend, when I was starting with programming and web development like 12 years ago, something like that. And I mostly, my biggest association with this phrase is how people started adding gradients and like completely change the whole style, like the stylistic way of how the web would work. So it wouldn't be frames anymore or images making, you know, the layout of the website, but it would be CSS, it will be cleaner and simpler. But then I guess what you're going after is the whole dynamic thing and how like WordPress and these other dynamic things popped up. But I guess that's not my main association with Web 2.0. It's mostly the progression to CSS and sort of cleaner, more modern style on the web. - Yeah, absolutely. And I guess my problem with the whole Web 1.0, 2.0 thing is it's a function of, there's a whole bunch of things that happened in the evolution of web design that sort of seem to have just overlapped on top of each other. And some people say, well, all that's what makes something web 2.0 or that's something that makes it web 1.0. But the truth is that it gets a bit blurry. And I know a lot of people debate it. And I guess, yeah, you're right. I mean, it's a static and dynamic thing is kind of what I was getting at. And the thing that makes it difficult is that web 1.0 was supposed to be more about just static sites, sites that never changed. You just, what you got is what you got. but the truth is that, you know, there were things called guest books way back in the day, you know, on your GeoCities page. And it was technically, that was a 2.0 feature 'cause a random user could, you know, write something on your guest book and then that would be displayed for everyone else to see. So technically that was a web 2.0 thing, but that was back in what was considered to be a web 1.0 in time period or environment. So it's not exactly clear cut. And I mean, I just want to, I guess, draw attention to the fact that, you know, it's, there was a migration away from this idea that it's good enough for your webpage to just be almost entirely static to going almost too far the other way where it has to be completely dynamic. In which case, you know, it's an interesting transition to consider. So anyway, not entirely the point, just wanted to bring it up and mention it 'cause it is one of those hotly debated, frustrating arguments, but anyway. - Right, in 2018, the whole Web 2.0 seems almost funny because it was just a phrase, it was not really a thing. It was a bunch of different things, as you said, overlapping in both design and programming side of things and then advancements in HTML and CSS and JavaScript and a bunch of people wanted to call it, give it a name. But from the perspective of time, it seems very meaningless. Like there was no step change. It was a bunch of different unrelated things coinciding over the span of really many years. - Yeah, exactly. And the funny thing is about Web 1.0 as well is that it was never known as Web 1.0 back in the day. It was one of those, yeah, a back acronym, not back acronym, you know what I mean? It was an expression that was created after the fact to refer to some point in time prior to the current point in time. It's kind of like you could say like when with Pragmatic, it's like Pragmatic version two was when it went indie or Pragmatic version three was when, yeah, it's a marker in time and a label that no one used at the time, but it has been come to represent something that mostly was this time period. - I think-- - So anyway. - I think the word you were looking for is retronym. - Ah, retronym, yes, thank you. That's exactly what I was looking for. - Just looked it up. I couldn't figure it out either. - That's cheating, looking things up. Anyway, oh, hang on. No, that's what I call research. Nevermind. All right, lovely. Okay, cool. So let's talk a little bit about then dynamic sites. And I guess the whole thing is that, and where I'm, my personal journey of frustration, I guess you could call it in web development, is the fact that when you're starting out, you tend to listen to different advice from different people. And the problem is that if they're not really thinking about this and framing this from the point of view of what's the best solution, they're thinking about what they've got experience with, they're thinking about, well, you could just do this, you could just do that. And it's not necessarily the best advice. And so my problem was that I went, I was very swayed by other people's opinions because I didn't have too much experience. So, you know, things like starting out with Blogger and then progressing all the way to where I am at the moment and all the points along that way. And I'm thinking back about that and I've gone about my approach to web development very, very differently. I know I would do it differently now because I'm now looking at, well, what are my requirements? What do I need to be dynamic and what don't I? Can I get away with static and so on and so forth? So I guess what I wanted to do really is today is like flesh out, what's the decision tree? How do you figure out which one you should use? And if you have to choose one or the other, what are your best options in either case and pros and cons and all that sort of thing. So with talking about dynamic sites then. So the great thing about a dynamic site is whenever a user accesses the site, it will specifically determine what to show the user based on the current state of the site and its data at that specific moment in time. And that can be fully dynamic or you can have elements of the site that are dynamic. And the data typically is stored in a database or it can be stored in a what's becoming, I suppose some people are preferring it like a flat file database, don't call it a database, but technically it's a database, just stored in a bunch of text files, which are slower to access and frustrating for different reasons, but they're more directly editable because people don't like editing databases, blah, blah, blah, blah, well, some people don't anyhow. Did I miss anything with dynamic sites? - No, no, let's keep going. - Okay, yeah, it's just one of those things, it's like when you think about it, that's really all there is to it, if you like peel it back, that's what it comes back to. And the difference with static sites is that they're always the same, no matter what, whoever accesses it, whenever they access it, it just doesn't change ever, ever, unless someone goes in and accesses the raw files and modifies the raw files. So the thing about, there's actually a new class of site, I think, and it's a statically generated site. And I look at statically generated sites as they're technically, yes, the output may be static, but technically they're not really static because static sites don't change unless the source files are changed, but the files that are served can change based on changes to the source file. So there's an inter, is an intermediate step and they can be made to appear to behave a bit like a dynamic site. If you use things like a recurring background task on the server to recreate the site periodically based on changes made to the source file. So it's not fully dynamic as in terms of a user's session when they log into the website, log in, when they access the website, you know what I mean? - Right. Technically, yeah. - And you can also add the dynamic features totally on the front end with JavaScript. So like, I don't know if it's still popular, but people used to use Disqus with like you, I think, to add comments to websites and there'll be like purely JavaScript. So, you know, really heavy and bad, I guess, for some reasons, but you could add these dynamic features or stuff like recommendations for related, you know, articles or something like that with while the pages on the server would be static. - Yeah, exactly. So I think that that's something that I have sort of dabbled with and actually JavaScript's been around for quite a while. So it's, and being able to execute code in the client to give that partly dynamic interaction has its place. But a lot of people don't really like JavaScript anymore. - Right. - I don't know. Anyway, so, all right. So let's see. All right, so on the dynamic side of, sorry, on the static side of things, just again, quickly. I guess the problem with static is it's difficult, it's difficult to do things like e-commerce, social media, lots of interactivity, it's really quite difficult. And I guess the idea of just anyone who wants to actually create a site, they need to be able to edit the HTML files on a web server, usually running Linux. And the problem with that is getting anyone who could actually create content meant that the people that created the content had to be people that were technically minded, that is to say, people like us. And you had to be a programmer or a developer to do anything on the web, or you had to have a team of people that did that for you. So I think that one of the things that really drove dynamic websites particularly was the idea of a content management system or a CMS, where you could actually get people that weren't technically savvy to essentially log into an administration panel. they could then go and create a file that would then eventually become a webpage and that webpage article would be based on the content that they created. But I didn't have to have any specific particular knowledge about how to edit HTML files or log on to using FTP or whatever, they didn't have to do any of that. All they had to do was just log into their CMS, create their content and away you go. And I see that as being certainly, in the space that I've definitely played in is that one of the drivers for dynamic websites? - Yeah, I would say the same thing, that even like the funny thing about static websites or static website generations, and I'm sure we'll get back to it, is that they seem simple, but only to really technical people who can deal with it. But then for everybody else, What made things like WordPress and these days, you know, Squarespace and such so popular is that you can really just play around with it and have your own website, even though you don't have technical skills. And that's something that will be quite difficult to achieve with static websites because it would have to be coupled with some sort of service to be able to upload it. And it would have to be like, you know, what you see is what you get. and we had back in the day, was it the Microsoft? Do you remember this? I don't remember how it's called, but there was this app, this WYSIWYG app for making websites, and it would produce the most bad, outrageous, unsemantic HTML that would be unreadable for search engines and such. I think, was it Microsoft Publisher or, no, it wasn't, there was another one. But yeah, I think I know the one you mean. And you would, yeah, you're right. It is the output. And you can actually do that even with Microsoft Word. You could actually push, you could push an output to HTML and it was completely, yeah, if you were to write it in raw HTML, just if you were to write it without the editor, it'd be one kilobyte. But if you exported it from this thing, it'd be like, you know, 25 kilobytes and it would just be full of rubbish in the HTML tags. And you'd be looking at it and you're thinking, how is that even possible to cram so many pointless HTML tags into this file? But it did manage to do it. It was, yeah, it was supposed to promise you that what you see is what you get kind of thing, but it never really delivered. But I did use it a couple of times and then I refused to use it ever again. - I just looked it up. It was called front page. - Oh, that's the one. Okay, cool. So, okay. So just then about the dynamic sites, actually, before we do that, I just wanna be clear. I didn't really wanna go necessarily too much into, you know, self, like self, quote unquote, self-hosted. That is to say you manage and control your own server and your own documents on the server or your own installation on the server versus whether it's a shared host or a virtual private server versus somewhere like Squarespace, which is essentially a website as a service kind of thing. So you go and they maintain their servers and they deal with all the uptime, downtime, all that other stuff. And you just have to log into a CMS and do what you wanna do. And you're more or less done. You can obviously hack a lot more and go down in templates and mess with things in Squarespace and certain other ones as well. And some people like that. So, you know, that's great. But beyond that, I guess I just wanna talk about the different kinds of dynamic sites that we've both played with, I guess. And I guess one of the great things about dynamic websites is that the database backend, you can store pretty much anything you want. In fact, practically everything, if you really want to, including the font that you wanna use and the pitch of the fonts and different styles, there's all sorts of stuff that you can throw in there, pretty much anything. And you just do a query to extract that for whatever purpose you need. And you can create a webpage that essentially lets other people enter that information through an admin panel. And there's lots of flexibility there. Ultimately though, there's a, I think the two probably most common database formats, MySQL and Postgres. I know there's more. Are there any other ones that you've used or come across? - Not really. I did play around with Ruby on Rails applications that would have in production SQLite websites, but it's only something that works for small websites. And then when you're serious, use MySQL, which is harder to set up, but just faster and more reliable for big websites. But mostly, yeah, MySQL is the most popular one. - So with MySQL recently, I recently did an install of CentOS 7 on a VPS and they've now renamed it to Maria. I didn't have a chance to really dig into that one actually, but I thought that was interesting. It's supposed to be, you know, instruction by instruction equivalent of MySQL, but I had some scripts I was trying to run for an RSS reader, an RSS aggregator, I should say. And yeah, it didn't like that. It was looking for MySQL specific instructions that weren't available in Mario, which was interesting and frustrating at the same time. - This is really interesting. I never heard about that. - Well, in any case, that's all good. So examples of websites that I've played with that were dynamic. The very first one I ever did was a blogger site. Technically, technically, it was a CMS. That was my first ever, actually no, that was my first ever, my first one was back in '97. This one was in 2008, I think it was. Anyway, and then after that, like everyone else I'm sure, WordPress. And I started out with WordPress as hosted on the WordPress website. And later on, I moved that to a self-hosted solution. And I bounced that around multiple different hosts over the years. And then after that, around about 2013 or 2014, I went to a Statomic, which is a flat file database, don't call it a database thing. And Statomic is a conjunction of two words, static and dynamic to create Statomic. And that was interesting. I started doing that for tech distortion and Statomic, the one I'm using is Statomic version one and V1 licenses, it's not free, right? So WordPress is free, Blogger was free. Obviously you pay to host it, the hosting is not free but the software is. And with Statomic, no it wasn't, it was a paid product, a commercial product. And it was originally, V1 was split into three components. You had the base license, if you wanted to use forms, they had a plugin called Raven Forms, and if you want to do search, they had a plugin called Bloodhound. So if you add all those licenses up, it was $197 US for a single site with all three licenses, which they essentially did with version two, and version two you buy it, that's in 199 US for a brand new license. Or if you've got a V1 license, you can get a $49 upgrade. Version three, which they're now working on, has been sort of nearly ready for release since about February this year, but it still isn't out yet, they're still working on it. They're saying it's gonna be a free upgrade for anyone that's on V2. But in any case, Statomic is an interesting product. It basically drove me to learn Markdown, which is, I guess, not a bad thing, but in the end, I also learned a lot of YAML and such for the front matter that you use for all your database don't call it database files. And anyway, so that's been my dynamic sites that I've played with. How about you? - All right, so my experience was with, I very briefly played with Tumblr, but never really published like a bigger real thing. There's something on the web, but like just pictures. I used WordPress for a while for many projects and like most people. And also there was my own CMS, which if you really dig deep, you can find somewhere on GitHub the source code for it. It was called Watermelon CMS. And that was, I guess, the first thing anybody learning PHP in 2006 would do, like as their first project, they would make a CMS. So instead of using something that was out there, I would build my own CMS to host my own website. But that didn't last very long. I think there's one website on the internet that's still hosted to this day on this thing. - Cool. - Yeah, and then after that, I hosted my new website, which is what you'll find still today on radix.io on Jekyll. So Jekyll is a static site generator. And I use it because I want something really simple, but something I like precisely. Like I wanted to write my own CSS to make it look exactly the way I want it. And Jekyll is this kind of thing that's going to be really complicated for a non-technical person. HTML and CSS and want to make your life easier than just using plain static website files, it's easier to just have Jekyll spit out exactly the websites, the web pages you want, than to work with WordPress and add plugins and change themes because there's so much stuff in WordPress that it's quite difficult and And for Jekyll, that's quite easy. And also one other thing I had experience with, the website for my podcast at thepodcast.fm is hosted at Squarespace, which is very typical of a podcast website. Yeah, and I'm fairly happy with that. I wouldn't, I don't think I would use it for my own website because I'm really picky about exactly how it works, but for this purpose it works quite well. My only annoyance with it is that you can't really use the like the admin part, the CMS part on an iPad or an iPhone. You really need a computer. Maybe that changed in the last year. So there's that and two things I had just brief interaction with but not too much direct experience is at my work at my job at Nozbe for a very long time. All company sites like Nozbe.com and and the CEO's website was hosted on a tiny static website generator built like 10 years ago by the CEO himself, by Michael. It was called Zenless. And it was, I guess, a little bit ahead of its time, but too primitive to keep using it. Because the way it worked is you would write posts or pages using Markdown, and you would save them just as markdown files in Dropbox and an automatic service would download changes from Dropbox, compile like actual HTML files and upload it to Amazon S3 or something like that. So for a very long time, this works like that. And that was very convenient for someone like Michael who really likes to work on his iPad because you could publish stuff and do everything from your iPad, which back in 2012, 2013, will be very difficult, like would be a lot, like would be impossible with something like Jekyll and not super convenient with something like WordPress or Squarespace. So there was that. And then we gave up on that because it was too primitive and it was too much hassle to extend for our new uses. And so we switched to another, to an open source static generator, which is called Hugo or Hugo, I don't know how to pronounce it. And I didn't know too much about it, except it's really powerful, way more powerful than Jekyll. So all of the company sites now are generated using that. And there's some really complicated stuff on the websites multiple languages, a lot of different pages, like posts are annotated with offers and co-offers and there's this whole structure and it's all like statically generated from markdown files that, you know, anyone in the company can edit. You don't have to be super technical to do it, you just have to be able to write markdown and then it's also hooked up so that it's, it's uploaded the generated website to the server. - Cool. Well, okay, so that's quite the list I've got to say. I personally just want to say, I have never rolled my own CMS and I don't ever want to roll my own CMS. And I know that there are people that have, I know that some other people that we know like Marco rolled his own for Marco.org, I'm pretty sure. Casey was did, his was called Camel. So now you've rolled your own and Michael rolled his own. That's yeah. So everyone's rolled their own CMS except John, but that's okay. I'm not gonna do that. I'm not gonna do it. But the thing with dynamic sites before we get into the statically generated ones, which I definitely wanna talk about is, I guess the problem that I have with dynamic sites in general. And I guess the first biggest problem I have with dynamic is caching. And when I say caching is like the caching problem. The problem is that if you were to execute, let's say it's written in PHP. So you've got to execute your PHP every single time. That's very, very processor intensive. And it's the sort of thing that it does not scale. And the more and more traffic you get, the load goes up and up and up until the point at which it just falls over. And so you have to do some level of caching. And as soon as you start doing caching, then you've got to figure out from a dynamic site, It's like, well, if I turn everything into a, if I generate it once and I store it in my cache indefinitely, well, that solves that problem. But then when I make changes, I have to bust that cache or break the cache or whatever you want to call it. And you have to trace that dependency to how exactly you break the cache when you do it. Do you use the file change time and date and the source files, or if you're changing a template file, you have to check that periodically to make sure that it hasn't changed. and then you've got to drive a regeneration of the cache. And then of course, do you regenerate the whole cache? Do you just regenerate a handful of files that you think are dependent upon the source files? And then of course, caching as it relates to query strings, for example, if you've got that sort of interaction happening, it gets very messy very quickly. And I guess what I have found also is that if you've got a lot of complexity in the site, and I had a lot of complexity in the Statimix sites where I'd written PHP, even parts of it that were native to Statomic that I never modified in the core API, I found that the execution time of that script could get quite long, like multiple seconds. And then as soon as the traffic went up, then it would spawn more and more threads to the point at which it basically, even at, I guess I'd call it a moderate scale, a low-end VPS simply couldn't do it. And I guess, yeah, that's been my experience with dynamic sites is that they simply, they don't scale. - Right, I haven't had the privilege to have that much traffic on my websites that I managed personally for this to be a problem. But I do know that I host, you know, radix.io and multiple other things on the cheapest possible hosting service that I could find. And it works, even when I had a pretty decent amount of traffic on one of the blog posts a couple of years ago, like it would make no difference because it's just serving static files. So you can have the cheapest possible service and you can have, you know, even, you know, hundreds or maybe thousands of requests per second. And even on the cheapest thing, it would just work because it's just serving plain, plain fast and it's really fast. Like, you know, I'm clicking between links and it's just there, which is not what I can say about most dynamic websites. - Yeah, exactly right. And this is kind of where I'm going with this is that the question that I never asked myself or rather I didn't understand enough about the difficulty with scaling and caching and was, I mean, did I actually need to have a dynamic site? And that was the question that I should have asked back in the beginning. So what happened with me was that I was talking to some friends that were building sites and the current trend, fashion, whatever you wanna call it, the in thing at that point was Statomic. And Statomic was and still remains to be pretty cool. but my problem was that I also took on a lot of extra burden when I started doing that. And I didn't realize it at the time. And it's been great as products go, but ultimately I've sort of hit the wall with it because when you're trying to squeeze every last bit of performance, it sort of reaches a limit as to how far you can go. And when I say caching, I guess I'm talking about this, that's caching as it relates to Stanimic. And I know that Stanimic V1 had issues with caching, so does V2 to an extent, but it's a lot better. it sort of reached that point where I'm like, well, caching is something that things like Apache, Nginx, you know, and I guess we'll talk a little bit about Varnish. You know, it's a solved problem in many respects. And it's the sort of thing that if you've got a static site, you sort of let either a reverse proxy or you let your web server actually deal with that for you. So you don't have to deal with it. And it's like, in that respect, it's a solved problem. But if you've got a dynamic site, that's another layer of caching on top of that. So with WordPress, for example, you can download a plugin called Super Cache, I think. And yeah, as I say, it was static, it has its own built into it. So the problem is that that's a layer of caching on top of a layer of caching, which is part of the reason why it becomes a problem, I think. - Right. That's not something you want to rely on too much. - No, I mean, you're asking for trouble the more layers of caching that you add because then it becomes, oh, I've made a change, it hasn't shown up on the website, So I've got to blow away which case, the top level case or the final case, or it just gets, yeah, gets complicated and gets frustrating. So I'm kind of done. I don't say done, not done done, but certainly mostly done with dynamic sites anyhow. And I'm slowly migrating stuff away from Statimix. So you mentioned some of the statically generated sites that you've played with. So I guess let's talk about statically generated sites now, 'cause I guess I kind of done with dynamics and moving on. And I guess the great thing about a statically generated site is that you can create a template or multiple templates for different, like you said, different languages, for example, and you can feed that different source files and issue a command through some mechanism, and it'll automatically regenerate the entire website of static files. And if you're using a cache on a server, you can tie that into the cache and have that trigger the cache busting command as well to trigger that, and you're good to go, basically, not much else to do. Advantages over a traditional site is you can safely edit that source data and the source files in something like Markdown. But there's not much risk that you'll mess up any of the HTML on the page because that's already generated for you, you've already sorted your templates out beforehand. And just referring back to when I started back my first webpage in 1997, that was an issue 'cause sometimes I do editing of the raw HTML files and I'd mess up like a square bracket, not a square bracket, a corner bracket, sorry. And my tags wouldn't line up and then the HTML wouldn't render properly. And it sounds like a silly little thing, but it was actually, it was a thing, you know, you could actually mess up the HTML on your page if you weren't careful because you were editing the mixture of the text as well as the tags. So by doing a statically generated site with all the templates and all the other stuff dealt with for you ahead of time, all you gotta do is worry about the source data going into it. So I think that's a big plus as well. So examples of websites where I've used that are static generators. So I also have had a play with Jekyll, but to be honest, I only ever really trialed it. I never went live with it. It was just a little bit of play around and fun while I was doing some investigating. The other thing that I've tried that's a static site generator, I'll mention again is Statomic. And that's one of the things that I sort of said before is a dynamic generator. Well, yes it is, but as of version 1.10, they added a static site generator. So you could actually generate a static site from your Statomic site. So it's sort of like you can use it in that mode if you want to, and you simply pointed at the static generator files when you serve it. And it's fine. But the problem with that was, when I was running it on the VPS it was running on, it kept dying. So I'd run the static side generator to get about a quarter of the way through and it would stop. And after a lot of digging, I realized that if I turned off NGINX during that time and waited 10 minutes, then it would finish. At which point I realized, No, that's really not very fast. So that was tech distortion. So tech distortion basically takes 10 minutes or thereabouts just under for Statomic to generate that as a static site, which is really quite horrific and your website's down during that time. So really not that good. I suppose that if I paid a lot of extra money and had a higher spec VPS, it would be a lot faster, but that's really not ideal. And the most recent one that I've played with and the one that I'm now slowly migrating my sites to is the one you mentioned before, which is Hugo, or sometimes people call it Go Hugo. And I'm pretty sure it is pronounced Hugo, but I actually haven't used it in a sentence with another developer before. So I'm gonna assume it's Hugo, but anyway. And I'm now using that for two of my sites and I'm looking to move them all over gradually over the next little while. And in difference, and sorry, with respect to Statomic and comparatively, if I'm running it on my Mac mini, which is a 2012 Mac mini, which is not particularly fast as machines go, it's a little Mac mini. Anyway, it takes three and a half seconds to actually generate the whole Tech Distortion website from source files. And the same site, exact same everything generated on the VPS that I use takes 1.125 seconds. So that's one and one eighth of a second. And that's what I'm talking about. - Nice. - Nice. So there's some other ones out there that's wanna mention exist. And this is the thing is that there seems to be a lot of them. And so the other ones, you got one called Hexo. There's one called Gatsby. And I found a website that listed, I guess you call them recognized multiple site, static site generators. And this is obviously excluding ones like Watermelon, I think that was yours, and Camel and whatever Marco called his, all these ones that people just rolled their own, that's not including any of them because I think the number would be in the tens of thousands otherwise. But in terms of multiple site recognized independent static site generators, the list was over 450 and it's climbing every single week. So there's definitely a lot of interest in this and it is gathering momentum, but the two most popular are Jekyll and GoHugo. So it's a good thing you've used Jekyll and I've used Hugo, so we can cover off those top two. So what makes a good static site generator, do you think? - I think that's going to depend on who you're going to ask. I think static site generators, these the way I understand them are mostly appealing to technical people, to programmers and so for programmers what would make a good static website generator is for it not to get in the way because that's what's going to entice you to use something like that something you know static and simple instead of something like WordPress that it doesn't get in the way that there's just not a lot of stuff but on the other hand you don't want to like when you you want to plug in things like you want a atom feed or you want, I don't know, whatever. The common things you might want to add that you don't have to do it from scratch, but you just plug in a couple of components. You don't write any of your own Ruby or Go or JavaScript code. You just write Markdown, hit a bunch of switches, and that's it. - I absolutely agree. And I think that it's the sort of thing that it's difficult to have a static site generator that is easy for the average person to use necessarily. But I think different aspects of, there are different things that make static site generators better or worse, I think, than others. So some are better than others. And I guess the first and foremost reason I can think of, you know, from a development point of view is the speed, the speed to generate the entire site. That's probably the biggest one. Because the last thing you want to do is to make a change, hit generate, and then go and get a coffee and come back in five minutes. I mean, that's not really what you want. You want something that's really, really fast to do that. And particularly, I think also the other sub requirement for that is it has to be very fast at rendering a local copy for local development. So if you run it up on your local machine, you hit save, how long does it take for that change to ripple through to the final product, which you can then look at in a browser, let's say. So I think that's key. And I also think that it's important with these static site generators that they're actively maintained. And I know you could argue that for a lot of open source stuff. So Jekyll is open source, if I remember correctly. Hugo is, yeah, Hugo is as well. And that community around actively maintaining them, that's really important because the problem with SSGs is that it's unlike something like WordPress where you can just like log in And if you wanna go with the basic defaults, you can, or you can tweak it to a point. There's a limit to how far you can go, but there's still a lot of people using it. And in the case of static site generators, obviously you need something that's actively maintained 'cause if you have got something that's actively maintained, then it's going to get a lot of support 'cause there's a community behind it. And that's important. I guess the other thing I think is really important as well is that there has to be good documentation. One of my only, I say only, one of my frustrations with GoHugo is that the documentation is not necessarily always up to date as you would like. And that's just because as a product goes, it's still relatively early on. I think it's up to 0.46 or 48, something like that. So it hasn't even hit technically 1.0 for whatever 1.0 is worth, but it's still very early in its lifespan, I guess you could say. It's only been around for a few years. So, the interesting thing is that I rely more on forums and looking at other people's examples than I actually get from the documentation. The documentation looks good, but sometimes it's so far out of date. It's getting better, but in any case. So, okay. So, specifically with Hugo, The thing that I found amazing when I first tried it was that you can actually generate static files in only a few seconds, even on a crummy Mac mini like mine. And that's in local memory. So you can actually run up a web server in local memory without any other tools. And so, 'cause normally I was used to running MAMP and MAMP, you know, so I can run up an NGINX or PHP environment that matches my server configuration. But I don't need that with Hugo. they could just be loaded up into memory. And if you then go to localhost port 1313, it'll just load directly from the memory on your local machine. And it basically waits and monitors for any file changes on the hard drive, sorry, solid state drive, whatever we're using these days. I think everyone's on SSDs by now, aren't they? So yes, spinning rust can go on and rust, please. Anyhow, and it'll actually look for file changes on the directory that you're executing it from. and if it detects any, it'll automatically regenerate the site and then that will boot the browser and the page will reload all pretty much automatically. It doesn't work every single time. Like sometimes I still have to manually refresh the webpage, but generally speaking, it works probably 95% of the time and it is ridiculously fast. I mean, I can make a change if I'm editing in Sublime Text, which is kind of my desktop editor of choice and I make a change to the source file, I hit save within a second, maybe two at the absolute outside, it'll regenerate all of those pages and it'll boot the browser and I'll see the result, which is fantastic. - I just checked with all of the Nozbe Hugo websites because you're right, I didn't mention speed because I was thinking of people making personal blogs or something like that. But when you have something big and complex, I remember one of the, perhaps the biggest reason why we chose Hugo was exactly that, that everything else was too slow. And I'm looking at the, I just hit the server. It was pretty easy to install. I did it just as we were explaining this and there's thousands of pages generated in 6.9 seconds. - Nice. - Yeah. - Well, the other thing that I think is also very good with static site generators, and I guess you could argue that Statomic is similar because it is flat file. and this is more of an advantage of flat file, but I think all static site generators are flat file by nature. But when you're in control of the site, you're able to use, if you like Git, for example, you can use a Git workflow for publishing. And that's also quite good. If you've got a lot of, I suppose, call them non-tech savvy people or users for the website, then you're probably gonna end up needing some kind of a CMS, I guess. That's the only downside. There are sort of ways and means you can sort of get around that. So there are some, you can actually create a, and there are editing solutions for editing like a file directory. You could potentially say, hey, here's how you create a new webpage and just show them how to use a basic tool for going to an FTP to a website. And you sort of like, the way I've done it with Bubble Sort, for example, is that I've loaded in a, it's a hidden URL specifically with PHP and then you log in with credentials and such to allow you to directly edit files in the actual content directory. So it's possible for other people to modify those. But generally speaking, even that requires some level of knowledge. It's not as pretty as a CMS. It does work, but you know, it's sort of like, hmm. So in any case, I guess the final advantage, want to mention quickly about static sites, whether they're statically generated or just static static static, which is not an expression but still, is security. And because there's no active scripts executing, there's no exploits really, well technically there's hardly any exploits, certainly none from the website generation itself. So it's about as safe as as you can pretty much get from those sorts of vulnerabilities. And there's no PHP or patching needed for anything like that, which is something you always got to stay on top of if you've got a dynamic site. - I was looking up Statamik earlier because I know that's what you are using. And on their website, they're quoting someone saying that, like a third or something like that of like website hacks or something like that happens on WordPress sites. And that's not surprising because when you have, like WordPress is a pretty good piece of software, but it's a really big piece of software. And there's always bugs and stuff like that. And there's bugs in PHP that are found too. And when you have something like that running publicly, then it's different when you, you know, when you have someone whose job it is to keep updating it and just maintain it. But most of the time you just want the website. Like you don't want a programmer or technical person to keep overseeing that you have absolutely everything most up to date. And whenever there's like a serious vulnerability discovered that you patch it like, you know, as fast as possible. With static websites, you don't have to worry about something like that. And I wanted to add one more just piece of data about the speed of Hugo. I checked and the less than seven seconds measured on the nozbe.com websites, that's for 23,000 files. - Wow. - Yeah. - That's impressive. - Most of it is like static files that are duplicated for multiple languages, but still, that's 23,000 files, thousands of which are actual pages in multiple languages. And that happens in seven seconds, which is not a lot. - That's pretty, it's pretty impressive. And comparatives with, you know, like Jekyll, for example, is that, you know, that could be anywhere from a quarter to a 10th of the amount of time to render the same thing in with Jekyll. But then the thing that, I guess, when I was reading up on Hugo, and it's funny actually that I was trying to figure out who it was or which tweet that I saw that mentioned Hugo. It was actually one of yours you'd mentioned that you guys were playing with it. And I sort of started digging into it and a few other people on Twitter had mentioned this as recently as like two years ago and it took me a while to give it a shot, but when I did, I was actually stunned at how much faster it was and I think it is the fastest currently, the fastest static site generator that there is it is pretty amazing actually how fast it is, but anyhow so one of the other things that I just want to mention regarding static site generators and static sites in general is that you can't just say across the board like I am going to go completely static because sometimes that's not always the best answer. There are always going to be well not always but I think that there are there's definitely scope for a hybrid solution where you do rely on it's a static site yes but you either have client-side execution or you can leverage existing third-party stuff if you want to So for example, when I moved Bubble Sort to GoHugo, I didn't want the show email account exposed to, you know, bots that are gonna crawl around and then start spamming the crap out of your poor email accounts. That's not what you want, which is a mistake I made early on, but nevermind that. So anyway, there's a web service that I use called 99Inbound for the form posting, and it works fine. So other than that, technically it's 100% static, but there is that one element. So if you need to have some kind of dynamic elements, you can look at doing something in the client, in the browser, or you can do something with a third party services. I guess the other thing is you could also potentially, if you want to, there's no reason why you couldn't enable PHP for a set of files or a series of files, rather than just making index PHP, the file that everything hits every single time. And that could be still useful to have some kind of a hybrid solution, depending upon what you're trying to achieve. but I don't pretend that static site generators are the answer to absolutely every problem. But if you're trying to reach a point where you've got a high performing site that can work on a low end server, so be economical and basically can be, your development can be extremely fast, then I think that static site generators are definitely worth serious consideration. - Yeah, I agree. If you have some technical skills to be able to deal with a tool like Jekyll for Hugo, then for a vast majority of use cases for a website, it's enough. Like you probably won't need dynamic stuff that you can't solve with JavaScript and it will save you a lot of pain. But I do think in the future, the approach of something like StataMaker, like some sort of hybrid approach might be really interesting. Like you said yourself that you hit limits of Statamix, the product as it is today, but an approach like this might be interesting in the future, where it's mostly statically generated and something fast, like Hugo's written in Go, so that's quite fast, but then you do have a CMS-like interface to be able to edit those things, so that it's more accessible to people who are not as technically inclined, And also so that you can plug in bits of dynamic behavior that's difficult to achieve otherwise. But still mostly rely on static generation so that you get something that's really fast and doesn't get in the way. - Yeah, I think you got a point there. And also compute power and especially, well, parallel compute power anyhow. It's not going up as fast as it used to, but it is still accelerating year over year. And for $5 a month now at Linode, for example, you can get a $5 US a month, you can get a reasonable entry-level VPS that is quite capable and can handle running Statomik at a moderate sort of scale. And in another two or three years time, that same $5 is gonna get you something that's more powerful again. So perhaps the whole execution problem and issues with caching get resolved to a point maybe it's because currently it's built on PHP so StaticMix built on PHP and it's Laravel is the is what they're using as the basis for it essentially a whole bunch of other plugins. The point is that it's quite possible that if that's rewritten and done on a higher performing you know language like Go then yes it's quite possible that that is the right answer in the end But for the moment at least, I'm... I guess you could say I'm in the honeymoon period with Hugo. And it's... yeah, I'm sure there's going to have issues with her further along. But for the moment at least, it's doing what I need. And the only other thing that I wanted to mention as well about just about caching is that a few years ago, I came across something called Varnish. And have you ever used Varnish at all? No, I did not. Okay, so Varnish is an open source, essentially it's a reverse proxy that handles caching between your web server, let's say it's Nginx or Apache, for example. And it was actually released in 2006 and a lot of people swore by it, they'd use that as their caching, essentially, like I say, it's acting as a reverse proxy and it's sort of, it's a one size fits all kind of a solution and it can be tailored quite specifically to different parts of a site. And I actually got how to play with that. And I also had, because I prefer Nginx over Apache, I have done for years. And so I was using Nginx and the FastCGI cache actually performs very similarly these days to Varnish. So when it came to the caching on, 'cause I still do, you still use the Nginx caching for the static sites. And it's just, it was a bit of a wash really. So I kind of gave up on Varnish 'cause Varnish was another necessary complexity at that point. It wasn't giving me any additional performance. So I just wanted to throw that in there because a lot of people sort of swore by Varnish and said, "Oh yeah, you gotta use Varnish and so on. "It's the best and everything." But I think that the other options that are out there that are built in, 'cause Nginx, obviously, it's just, if I added Varnish on top of Nginx, then I've gotta maintain both Nginx as well as Varnish. It takes memory, blah, blah, blah. So you're just better off, I think, anyway, just sticking with the Nginx caching. So that's been my experience anyhow. And that applies to static or dynamic sites. But I also had issues with caching as it related to, like I said, two levels of caching. So with Statomic, so you had the caching in Statomic, then you had the caching in NGINX. And then when I was using Varnish, you only had caching in Varnish. So you had three levels of caching. And it was, yeah, it would have some bizarre behavior where sometimes you had to seriously bust every single one of them in order for it to actually show, push a change out, which was very frustrating. Whereas with Hugo, you just statically generate it and the nginx cache is the only cache and that's it. And I haven't had any issues with it at all. And what I do with Hugo is that I have a cron job that executes every five minutes and it just regenerates the site. So if I make changes to the source files, every five minutes, it just re crunches those and it automatically syncs up and publishes that and it goes live. So I can actually schedule a file to go live at a certain time and date. You can also do like conditional stuff like most of these, you know, static site generators. You can actually specify a time to publish and a time to unpublish if you want to. So if you want something on the site that's time limited, you can do that as well. And as time moves forward, it just automatically generates the site and it displays what you would expect it to display, which is really great. So it kind of looks like a bit of a dynamic site, but in truth, it's not. So I think that's fantastic. I'm not surprised at all to hear your caching problems. There's a saying or a joke, I guess, among programmers that the three most difficult problems in computing are off by one errors, naming things, and cache invalidation. And when you have to use a cache, that often indicates a bigger problem that some piece of computation that should be fast is not. And if you can, if it's practical to just avoid the problem altogether by skipping the computation or changing it to something that's just really fast and doesn't need a cache, you're going to save yourself a ton of headache over the long term. And that's like, we're talking about, you know, static site generators or any other thing in computing. You just time and time and again, you get really bitten by caches all the time. - Awesome. Alrighty, well, if you wanna talk more about this, you can reach me on mastodon@[email protected], or you can follow @engineered_net on Twitter to see show-related announcements. If you're enjoying Pragmatic and you wanna support the show, you can, like some of our backers, Karsten Hansen and John Whitlow. They and many others are patrons of the show via Patreon, and you can find it at patreon.com/johnchijji, or one word. Patreon rewards include a named thank you on the website, a named thank you at the end of episodes, access to pages of raw show notes, as well as ad-free high quality releases of every episode. So if you'd like to contribute something, anything at all, there's lots of great rewards and beyond that, it's all very much appreciated. If you're not in a position to support the show via Patreon, that's fine. You can still help by leaving a rating in iTunes, favoriting the episode, or the show in your podcast player app of choice, where it's supported, or by sharing the episode or show on social media. It helps other people find out about the show and that's all very much appreciated as well. Pragmatic is part of the Engineer Network and you can find it at engineer.network along with other great shows like Causality, which is a solo podcast I do that looks at cause and effect of major events and disasters in history, including Three Mile Island, the Challenger Space Shuttle, and recently Chernobyl and lots more. Causality is on track to overtake Pragmatic as it grows in popularity. So if you haven't tried it out yet, be sure you do. Now, if you'd like to get in touch with Radek, what's the best way for people to get in touch with you, mate? So you can find me on Twitter @radxp, R-A-D-E-X-P. You can listen to me and my friend Michael every week on the podcast, thepodcast.fm. And I blog about programming at radx.io. - Fantastic, all right, cool. Well, a special thank you again to our patrons and a big thank you to everyone for listening and thank you for coming back on the show, Radek. It's been great. - Thank you so much for inviting me again. And I wanted to also really recommend Causality. I really dig that show. - Oh, cool. - Thank you very much. (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) (upbeat music) [Music] (dramatic music)