Welcome to Pragmatic.
Pragmatic is a discussion show contemplating the practical application of technology.
By exploring the real-world trade-offs, we look at how great ideas are transformed into products and services that can change our lives.
Nothing is as simple as it seems.
Pragmatic is part of the Engineered Network. To support our shows including this one, head over to our Patreon page.
And for other great shows, visit engineered.network today.
I'm your host, John Chidji, and today I'm joined by Radek Pietruszewski.
Did I get that right this time?
Yes, you did. Hello!
Alright. Hello! Thanks for coming back on the show.
Thank you for inviting me again.
- Yeah, no worries.
I've been wanting to talk about a topic
particularly recently because of my recent experiences
and my, I guess you could call it my journey
as a web developer, which I'm really not a web developer,
but I kind of pretend to be sometimes in my spare time.
And this is the argument, I think you call it,
I'd like to say it's an age old argument,
but it's not really that old as arguments go,
this whole static versus dynamic thing.
So I was, I kinda, yeah,
it's been an interesting, I guess, journey.
I don't know if you'd call it that for me.
And I was hoping you can help me
sort of flesh this one out a little bit.
- All right, let's try and do that.
- Alrighty, so way back in the beginning,
which let's be honest was really only a few decades ago,
like websites used to be pretty simple.
a bunch of images, maybe an animated image,
if you're really fancy, a bit of HTML,
some HTML tags around the text,
maybe a couple of files or two, maybe.
I think we got PDFs back then and different things
and you were done.
But websites had to be developed by people
that understood how to FTP onto a server
and do all those modifications
to the raw files and everything.
And the first website I ever did was exactly that.
And I actually had an animated Australian flag
blowing in a virtual breeze,
which thinking back was really tacky.
But at the time I thought it was just amazing,
but you know, it was, yeah, anyway.
But it was fully static.
And if you wanted people to email you, that was fine.
You know, you just put your email address on the website.
'Cause I mean, nothing could go wrong if you did that, right?
Just put your email address there, that's fine.
Job done.
And if you wanted to have some interaction,
you know, you could do forms and basic forms
that came along with no real encryption or anything.
You just post all the details and away you go.
What could go wrong with that either?
And when I was actually first working on forms
in particular, I was doing some work at Nortel
and they had me doing some Perl scripts in the backend
for some CGI bin forms back in '97.
That was never externally facing,
it was internal on Nortel's intranet.
And that was all very interesting, I guess.
slanty toothpick and all that other thing.
And time moves on and the world moves on
and websites move on.
And I guess this is where we start to delve
into the whole web 1.0 versus web 2.0 thing.
So when you think of web 1.0 to 2.0,
how do you think of that transition?
Like which bucket is which, would you say?
- That's a really good one.
I recall Web 2.0 being a thing, a trend, when I was starting with programming and web development
like 12 years ago, something like that.
And I mostly, my biggest association with this phrase is how people started adding gradients
and like completely change the whole style,
like the stylistic way of how the web would work.
So it wouldn't be frames anymore
or images making, you know, the layout of the website,
but it would be CSS, it will be cleaner and simpler.
But then I guess what you're going after
is the whole dynamic thing and how like WordPress
and these other dynamic things popped up.
But I guess that's not my main association with Web 2.0.
It's mostly the progression to CSS
and sort of cleaner, more modern style on the web.
- Yeah, absolutely.
And I guess my problem with the whole Web 1.0, 2.0 thing
is it's a function of,
there's a whole bunch of things that happened
in the evolution of web design
that sort of seem to have just overlapped
on top of each other.
And some people say, well,
all that's what makes something web 2.0
or that's something that makes it web 1.0.
But the truth is that it gets a bit blurry.
And I know a lot of people debate it.
And I guess, yeah, you're right.
I mean, it's a static and dynamic thing
is kind of what I was getting at.
And the thing that makes it difficult
is that web 1.0 was supposed to be more
about just static sites, sites that never changed.
You just, what you got is what you got.
but the truth is that, you know,
there were things called guest books way back in the day,
you know, on your GeoCities page.
And it was technically, that was a 2.0 feature
'cause a random user could, you know,
write something on your guest book
and then that would be displayed for everyone else to see.
So technically that was a web 2.0 thing,
but that was back in what was considered to be a web 1.0
in time period or environment.
So it's not exactly clear cut.
And I mean, I just want to, I guess, draw attention
to the fact that, you know,
it's, there was a migration away from this idea
that it's good enough for your webpage
to just be almost entirely static
to going almost too far the other way
where it has to be completely dynamic.
In which case, you know,
it's an interesting transition to consider.
So anyway, not entirely the point,
just wanted to bring it up and mention it
'cause it is one of those hotly debated,
frustrating arguments, but anyway.
- Right, in 2018, the whole Web 2.0 seems almost funny
because it was just a phrase, it was not really a thing.
It was a bunch of different things, as you said,
overlapping in both design and programming side of things
and then advancements in HTML and CSS and JavaScript
and a bunch of people wanted to call it, give it a name.
But from the perspective of time,
it seems very meaningless.
Like there was no step change.
It was a bunch of different unrelated things
coinciding over the span of really many years.
- Yeah, exactly.
And the funny thing is about Web 1.0 as well
is that it was never known as Web 1.0 back in the day.
It was one of those, yeah, a back acronym,
not back acronym, you know what I mean?
It was an expression that was created after the fact
to refer to some point in time
prior to the current point in time.
It's kind of like you could say like when with Pragmatic,
it's like Pragmatic version two was when it went indie
or Pragmatic version three was when,
yeah, it's a marker in time and a label
that no one used at the time,
but it has been come to represent something
that mostly was this time period.
- I think-- - So anyway.
- I think the word you were looking for is retronym.
- Ah, retronym, yes, thank you.
That's exactly what I was looking for.
- Just looked it up.
I couldn't figure it out either.
- That's cheating, looking things up.
Anyway, oh, hang on.
No, that's what I call research.
Nevermind.
All right, lovely.
Okay, cool.
So let's talk a little bit about then dynamic sites.
And I guess the whole thing is that,
and where I'm, my personal journey of frustration,
I guess you could call it in web development,
is the fact that when you're starting out,
you tend to listen to different advice from different people.
And the problem is that if they're not really thinking
about this and framing this from the point of view
of what's the best solution,
they're thinking about what they've got experience with,
they're thinking about, well, you could just do this,
you could just do that.
And it's not necessarily the best advice.
And so my problem was that I went,
I was very swayed by other people's opinions
because I didn't have too much experience.
So, you know, things like starting out with Blogger
and then progressing all the way to where I am at the moment
and all the points along that way.
And I'm thinking back about that
and I've gone about my approach to web development
very, very differently.
I know I would do it differently now
because I'm now looking at, well, what are my requirements?
What do I need to be dynamic and what don't I?
Can I get away with static and so on and so forth?
So I guess what I wanted to do really is today
is like flesh out, what's the decision tree?
How do you figure out which one you should use?
And if you have to choose one or the other,
what are your best options in either case
and pros and cons and all that sort of thing.
So with talking about dynamic sites then.
So the great thing about a dynamic site
is whenever a user accesses the site,
it will specifically determine what to show the user
based on the current state of the site
and its data at that specific moment in time.
And that can be fully dynamic
or you can have elements of the site that are dynamic.
And the data typically is stored in a database
or it can be stored in a what's becoming,
I suppose some people are preferring it
like a flat file database, don't call it a database,
but technically it's a database,
just stored in a bunch of text files,
which are slower to access
and frustrating for different reasons,
but they're more directly editable
because people don't like editing databases,
blah, blah, blah, blah,
well, some people don't anyhow.
Did I miss anything with dynamic sites?
- No, no, let's keep going.
- Okay, yeah, it's just one of those things,
it's like when you think about it,
that's really all there is to it,
if you like peel it back, that's what it comes back to.
And the difference with static sites is that
they're always the same, no matter what,
whoever accesses it, whenever they access it,
it just doesn't change ever, ever,
unless someone goes in and accesses the raw files
and modifies the raw files.
So the thing about,
there's actually a new class of site, I think,
and it's a statically generated site.
And I look at statically generated sites
as they're technically, yes, the output may be static,
but technically they're not really static
because static sites don't change
unless the source files are changed,
but the files that are served can change
based on changes to the source file.
So there's an inter, is an intermediate step
and they can be made to appear to behave
a bit like a dynamic site.
If you use things like a recurring background task
on the server to recreate the site periodically
based on changes made to the source file.
So it's not fully dynamic as in terms of a user's session
when they log into the website, log in,
when they access the website, you know what I mean?
- Right.
Technically, yeah.
- And you can also add the dynamic features
totally on the front end with JavaScript.
So like, I don't know if it's still popular,
but people used to use Disqus with like you, I think,
to add comments to websites
and there'll be like purely JavaScript.
So, you know, really heavy and bad, I guess,
for some reasons, but you could add these dynamic features
or stuff like recommendations for related,
you know, articles or something like that
with while the pages on the server would be static.
- Yeah, exactly.
So I think that that's something that I have sort of
dabbled with and actually JavaScript's been around
for quite a while.
So it's, and being able to execute code in the client
to give that partly dynamic interaction has its place.
But a lot of people don't really like JavaScript anymore.
- Right.
- I don't know.
Anyway, so, all right.
So let's see.
All right, so on the dynamic side of,
sorry, on the static side of things, just again, quickly.
I guess the problem with static is it's difficult,
it's difficult to do things like e-commerce,
social media, lots of interactivity,
it's really quite difficult.
And I guess the idea of just anyone
who wants to actually create a site,
they need to be able to edit the HTML files
on a web server, usually running Linux.
And the problem with that is getting anyone
who could actually create content
meant that the people that created the content
had to be people that were technically minded,
that is to say, people like us.
And you had to be a programmer or a developer
to do anything on the web,
or you had to have a team of people that did that for you.
So I think that one of the things
that really drove dynamic websites particularly
was the idea of a content management system or a CMS,
where you could actually get people
that weren't technically savvy
to essentially log into an administration panel.
they could then go and create a file
that would then eventually become a webpage
and that webpage article would be based on the content
that they created.
But I didn't have to have any specific particular knowledge
about how to edit HTML files or log on to using FTP
or whatever, they didn't have to do any of that.
All they had to do was just log into their CMS,
create their content and away you go.
And I see that as being certainly,
in the space that I've definitely played in
is that one of the drivers for dynamic websites?
- Yeah, I would say the same thing,
that even like the funny thing about static websites
or static website generations,
and I'm sure we'll get back to it,
is that they seem simple,
but only to really technical people who can deal with it.
But then for everybody else,
What made things like WordPress and these days,
you know, Squarespace and such so popular is that
you can really just play around with it
and have your own website,
even though you don't have technical skills.
And that's something that will be quite difficult
to achieve with static websites
because it would have to be coupled with
some sort of service to be able to upload it.
And it would have to be like,
you know, what you see is what you get.
and we had back in the day,
was it the Microsoft?
Do you remember this?
I don't remember how it's called,
but there was this app, this WYSIWYG app
for making websites,
and it would produce the most bad,
outrageous, unsemantic HTML
that would be unreadable for search engines and such.
I think, was it Microsoft Publisher or,
no, it wasn't, there was another one.
But yeah, I think I know the one you mean.
And you would, yeah, you're right.
It is the output.
And you can actually do that even with Microsoft Word.
You could actually push,
you could push an output to HTML and it was completely,
yeah, if you were to write it in raw HTML,
just if you were to write it without the editor,
it'd be one kilobyte.
But if you exported it from this thing,
it'd be like, you know, 25 kilobytes
and it would just be full of rubbish in the HTML tags.
And you'd be looking at it and you're thinking,
how is that even possible to cram
so many pointless HTML tags into this file?
But it did manage to do it.
It was, yeah, it was supposed to promise you
that what you see is what you get kind of thing,
but it never really delivered.
But I did use it a couple of times
and then I refused to use it ever again.
- I just looked it up.
It was called front page.
- Oh, that's the one.
Okay, cool.
So, okay.
So just then about the dynamic sites,
actually, before we do that, I just wanna be clear.
I didn't really wanna go necessarily too much into,
you know, self, like self, quote unquote, self-hosted.
That is to say you manage and control your own server
and your own documents on the server
or your own installation on the server
versus whether it's a shared host
or a virtual private server
versus somewhere like Squarespace,
which is essentially a website as a service kind of thing.
So you go and they maintain their servers
and they deal with all the uptime, downtime,
all that other stuff.
And you just have to log into a CMS
and do what you wanna do.
And you're more or less done.
You can obviously hack a lot more
and go down in templates and mess with things
in Squarespace and certain other ones as well.
And some people like that.
So, you know, that's great.
But beyond that, I guess I just wanna talk
about the different kinds of dynamic sites
that we've both played with, I guess.
And I guess one of the great things about dynamic websites
is that the database backend,
you can store pretty much anything you want.
In fact, practically everything, if you really want to,
including the font that you wanna use
and the pitch of the fonts and different styles,
there's all sorts of stuff that you can throw in there,
pretty much anything.
And you just do a query to extract that
for whatever purpose you need.
And you can create a webpage that essentially
lets other people enter that information
through an admin panel.
And there's lots of flexibility there.
Ultimately though, there's a,
I think the two probably most common database formats,
MySQL and Postgres.
I know there's more.
Are there any other ones that you've used or come across?
- Not really.
I did play around with Ruby on Rails applications
that would have in production SQLite websites,
but it's only something that works for small websites.
And then when you're serious, use MySQL,
which is harder to set up,
but just faster and more reliable for big websites.
But mostly, yeah, MySQL is the most popular one.
- So with MySQL recently,
I recently did an install of CentOS 7 on a VPS
and they've now renamed it to Maria.
I didn't have a chance to really dig into that one actually,
but I thought that was interesting.
It's supposed to be, you know,
instruction by instruction equivalent of MySQL,
but I had some scripts I was trying to run
for an RSS reader, an RSS aggregator, I should say.
And yeah, it didn't like that.
It was looking for MySQL specific instructions
that weren't available in Mario,
which was interesting and frustrating at the same time.
- This is really interesting.
I never heard about that.
- Well, in any case, that's all good.
So examples of websites that I've played with
that were dynamic.
The very first one I ever did was a blogger site.
Technically, technically, it was a CMS.
That was my first ever, actually no,
that was my first ever, my first one was back in '97.
This one was in 2008, I think it was.
Anyway, and then after that,
like everyone else I'm sure, WordPress.
And I started out with WordPress
as hosted on the WordPress website.
And later on, I moved that to a self-hosted solution.
And I bounced that around multiple different hosts
over the years.
And then after that, around about 2013 or 2014,
I went to a Statomic, which is a flat file database,
don't call it a database thing.
And Statomic is a conjunction of two words,
static and dynamic to create Statomic.
And that was interesting.
I started doing that for tech distortion
and Statomic, the one I'm using is Statomic version one
and V1 licenses, it's not free, right?
So WordPress is free, Blogger was free.
Obviously you pay to host it, the hosting is not free
but the software is.
And with Statomic, no it wasn't,
it was a paid product, a commercial product.
And it was originally, V1 was split into three components.
You had the base license, if you wanted to use forms,
they had a plugin called Raven Forms,
and if you want to do search,
they had a plugin called Bloodhound.
So if you add all those licenses up,
it was $197 US for a single site with all three licenses,
which they essentially did with version two,
and version two you buy it,
that's in 199 US for a brand new license.
Or if you've got a V1 license,
you can get a $49 upgrade.
Version three, which they're now working on,
has been sort of nearly ready for release
since about February this year,
but it still isn't out yet, they're still working on it.
They're saying it's gonna be a free upgrade
for anyone that's on V2.
But in any case, Statomic is an interesting product.
It basically drove me to learn Markdown,
which is, I guess, not a bad thing,
but in the end, I also learned a lot of YAML
and such for the front matter that you use
for all your database don't call it database files.
And anyway, so that's been my dynamic sites
that I've played with.
How about you?
- All right, so my experience was with,
I very briefly played with Tumblr,
but never really published like a bigger real thing.
There's something on the web, but like just pictures.
I used WordPress for a while for many projects
and like most people.
And also there was my own CMS,
which if you really dig deep,
you can find somewhere on GitHub the source code for it.
It was called Watermelon CMS.
And that was, I guess, the first thing
anybody learning PHP in 2006 would do,
like as their first project, they would make a CMS.
So instead of using something that was out there,
I would build my own CMS to host my own website.
But that didn't last very long.
I think there's one website on the internet
that's still hosted to this day on this thing.
- Cool.
- Yeah, and then after that,
I hosted my new website,
which is what you'll find still today on radix.io on Jekyll.
So Jekyll is a static site generator.
And I use it because I want something really simple, but something I like precisely.
Like I wanted to write my own CSS to make it look exactly the way I want it.
And Jekyll is this kind of thing that's going to be really complicated for a non-technical
person.
HTML and CSS and want to make your life easier than just using plain static
website files, it's easier to just have Jekyll spit out exactly the websites, the
web pages you want, than to work with WordPress and add plugins and change
themes because there's so much stuff in WordPress that it's quite difficult and
And for Jekyll, that's quite easy.
And also one other thing I had experience with,
the website for my podcast at thepodcast.fm
is hosted at Squarespace,
which is very typical of a podcast website.
Yeah, and I'm fairly happy with that.
I wouldn't, I don't think I would use it for my own website
because I'm really picky about exactly how it works, but for this purpose it
works quite well. My only annoyance with it is that you can't really use the
like the admin part, the CMS part on an iPad or an iPhone. You really need a
computer. Maybe that changed in the last year. So there's that and two things I
had just brief interaction with but not too much direct experience is at my work
at my job at Nozbe for a very long time. All company sites like Nozbe.com and
and the CEO's website was hosted on a tiny static website
generator built like 10 years ago by the CEO himself,
by Michael.
It was called Zenless.
And it was, I guess, a little bit ahead of its time,
but too primitive to keep using it.
Because the way it worked is you would
write posts or pages using Markdown,
and you would save them just as markdown files in Dropbox
and an automatic service would download changes
from Dropbox, compile like actual HTML files
and upload it to Amazon S3 or something like that.
So for a very long time, this works like that.
And that was very convenient for someone like Michael
who really likes to work on his iPad
because you could publish stuff and do everything
from your iPad, which back in 2012, 2013,
will be very difficult,
like would be a lot,
like would be impossible with something like Jekyll
and not super convenient with something like WordPress
or Squarespace.
So there was that.
And then we gave up on that because it was too primitive
and it was too much hassle to extend for our new uses.
And so we switched to another,
to an open source static generator,
which is called Hugo or Hugo,
I don't know how to pronounce it.
And I didn't know too much about it,
except it's really powerful, way more powerful than Jekyll.
So all of the company sites now are generated using that.
And there's some really complicated stuff on the websites
multiple languages, a lot of different pages, like posts are annotated with
offers and co-offers and there's this whole structure and it's all like
statically generated from markdown files that, you know, anyone in the company can
edit. You don't have to be super technical to do it, you just
have to be able to write markdown and then it's also hooked up so that it's,
it's uploaded the generated website to the server.
- Cool.
Well, okay, so that's quite the list I've got to say.
I personally just want to say,
I have never rolled my own CMS
and I don't ever want to roll my own CMS.
And I know that there are people that have,
I know that some other people that we know
like Marco rolled his own for Marco.org, I'm pretty sure.
Casey was did, his was called Camel.
So now you've rolled your own and Michael rolled his own.
That's yeah.
So everyone's rolled their own CMS except John,
but that's okay.
I'm not gonna do that.
I'm not gonna do it.
But the thing with dynamic sites
before we get into the statically generated ones,
which I definitely wanna talk about is,
I guess the problem that I have
with dynamic sites in general.
And I guess the first biggest problem
I have with dynamic is caching.
And when I say caching is like the caching problem.
The problem is that if you were to execute,
let's say it's written in PHP.
So you've got to execute your PHP every single time.
That's very, very processor intensive.
And it's the sort of thing that it does not scale.
And the more and more traffic you get,
the load goes up and up and up
until the point at which it just falls over.
And so you have to do some level of caching.
And as soon as you start doing caching,
then you've got to figure out from a dynamic site,
It's like, well, if I turn everything into a,
if I generate it once and I store it in my cache
indefinitely, well, that solves that problem.
But then when I make changes, I have to bust that cache
or break the cache or whatever you want to call it.
And you have to trace that dependency
to how exactly you break the cache when you do it.
Do you use the file change time and date
and the source files,
or if you're changing a template file,
you have to check that periodically
to make sure that it hasn't changed.
and then you've got to drive a regeneration of the cache.
And then of course, do you regenerate the whole cache?
Do you just regenerate a handful of files
that you think are dependent upon the source files?
And then of course, caching as it relates to query strings,
for example, if you've got that sort of
interaction happening, it gets very messy very quickly.
And I guess what I have found also is that
if you've got a lot of complexity in the site,
and I had a lot of complexity in the Statimix sites
where I'd written PHP, even parts of it
that were native to Statomic
that I never modified in the core API,
I found that the execution time of that script
could get quite long, like multiple seconds.
And then as soon as the traffic went up,
then it would spawn more and more threads
to the point at which it basically,
even at, I guess I'd call it a moderate scale,
a low-end VPS simply couldn't do it.
And I guess, yeah, that's been my experience
with dynamic sites is that they simply, they don't scale.
- Right, I haven't had the privilege
to have that much traffic on my websites
that I managed personally for this to be a problem.
But I do know that I host, you know, radix.io
and multiple other things on the cheapest possible
hosting service that I could find.
And it works, even when I had a pretty decent amount
of traffic on one of the blog posts a couple of years ago,
like it would make no difference
because it's just serving static files.
So you can have the cheapest possible service
and you can have, you know, even, you know,
hundreds or maybe thousands of requests per second.
And even on the cheapest thing, it would just work
because it's just serving plain,
plain fast and it's really fast.
Like, you know, I'm clicking between links
and it's just there, which is not what I can say
about most dynamic websites.
- Yeah, exactly right.
And this is kind of where I'm going with this
is that the question that I never asked myself
or rather I didn't understand enough about the difficulty
with scaling and caching and was, I mean,
did I actually need to have a dynamic site?
And that was the question that I should have asked
back in the beginning.
So what happened with me was that I was talking
to some friends that were building sites
and the current trend, fashion,
whatever you wanna call it, the in thing
at that point was Statomic.
And Statomic was and still remains to be pretty cool.
but my problem was that I also took on a lot of extra burden
when I started doing that.
And I didn't realize it at the time.
And it's been great as products go,
but ultimately I've sort of hit the wall with it
because when you're trying to squeeze
every last bit of performance,
it sort of reaches a limit as to how far you can go.
And when I say caching, I guess I'm talking about this,
that's caching as it relates to Stanimic.
And I know that Stanimic V1 had issues with caching,
so does V2 to an extent, but it's a lot better.
it sort of reached that point where I'm like, well,
caching is something that things like Apache, Nginx,
you know, and I guess we'll talk a little bit about Varnish.
You know, it's a solved problem in many respects.
And it's the sort of thing that if you've got a static site,
you sort of let either a reverse proxy
or you let your web server actually deal with that for you.
So you don't have to deal with it.
And it's like, in that respect, it's a solved problem.
But if you've got a dynamic site,
that's another layer of caching on top of that.
So with WordPress, for example,
you can download a plugin called Super Cache, I think.
And yeah, as I say, it was static,
it has its own built into it.
So the problem is that that's a layer of caching
on top of a layer of caching,
which is part of the reason why it becomes a problem,
I think.
- Right.
That's not something you want to rely on too much.
- No, I mean, you're asking for trouble
the more layers of caching that you add
because then it becomes,
oh, I've made a change, it hasn't shown up on the website,
So I've got to blow away which case,
the top level case or the final case,
or it just gets, yeah,
gets complicated and gets frustrating.
So I'm kind of done.
I don't say done, not done done,
but certainly mostly done with dynamic sites anyhow.
And I'm slowly migrating stuff away from Statimix.
So you mentioned some of the statically generated sites
that you've played with.
So I guess let's talk about statically generated sites now,
'cause I guess I kind of done with dynamics and moving on.
And I guess the great thing
about a statically generated site is that you can create
a template or multiple templates for different,
like you said, different languages, for example,
and you can feed that different source files
and issue a command through some mechanism,
and it'll automatically regenerate the entire website
of static files.
And if you're using a cache on a server,
you can tie that into the cache and have that trigger
the cache busting command as well to trigger that,
and you're good to go, basically, not much else to do.
Advantages over a traditional site is you can safely edit
that source data and the source files
in something like Markdown.
But there's not much risk
that you'll mess up any of the HTML on the page
because that's already generated for you,
you've already sorted your templates out beforehand.
And just referring back to when I started back
my first webpage in 1997,
that was an issue 'cause sometimes I do editing
of the raw HTML files
and I'd mess up like a square bracket,
not a square bracket, a corner bracket, sorry.
And my tags wouldn't line up
and then the HTML wouldn't render properly.
And it sounds like a silly little thing,
but it was actually, it was a thing, you know,
you could actually mess up the HTML on your page
if you weren't careful because you were editing
the mixture of the text as well as the tags.
So by doing a statically generated site
with all the templates and all the other stuff
dealt with for you ahead of time,
all you gotta do is worry about the source data
going into it.
So I think that's a big plus as well.
So examples of websites where I've used
that are static generators.
So I also have had a play with Jekyll,
but to be honest, I only ever really trialed it.
I never went live with it.
It was just a little bit of play around and fun
while I was doing some investigating.
The other thing that I've tried
that's a static site generator,
I'll mention again is Statomic.
And that's one of the things that I sort of said before
is a dynamic generator.
Well, yes it is, but as of version 1.10,
they added a static site generator.
So you could actually generate a static site
from your Statomic site.
So it's sort of like you can use it in that mode
if you want to, and you simply pointed
at the static generator files when you serve it.
And it's fine.
But the problem with that was,
when I was running it on the VPS it was running on,
it kept dying.
So I'd run the static side generator
to get about a quarter of the way through
and it would stop.
And after a lot of digging,
I realized that if I turned off NGINX during that time
and waited 10 minutes, then it would finish.
At which point I realized,
No, that's really not very fast.
So that was tech distortion.
So tech distortion basically takes 10 minutes
or thereabouts just under for Statomic
to generate that as a static site,
which is really quite horrific and your website's down
during that time.
So really not that good.
I suppose that if I paid a lot of extra money
and had a higher spec VPS, it would be a lot faster,
but that's really not ideal.
And the most recent one that I've played with
and the one that I'm now slowly migrating my sites to
is the one you mentioned before, which is Hugo,
or sometimes people call it Go Hugo.
And I'm pretty sure it is pronounced Hugo,
but I actually haven't used it in a sentence
with another developer before.
So I'm gonna assume it's Hugo, but anyway.
And I'm now using that for two of my sites
and I'm looking to move them all over gradually
over the next little while.
And in difference, and sorry,
with respect to Statomic and comparatively,
if I'm running it on my Mac mini,
which is a 2012 Mac mini,
which is not particularly fast as machines go,
it's a little Mac mini.
Anyway, it takes three and a half seconds
to actually generate the whole Tech Distortion website
from source files.
And the same site, exact same everything
generated on the VPS that I use takes 1.125 seconds.
So that's one and one eighth of a second.
And that's what I'm talking about.
- Nice. - Nice.
So there's some other ones out there
that's wanna mention exist.
And this is the thing is that
there seems to be a lot of them.
And so the other ones, you got one called Hexo.
There's one called Gatsby.
And I found a website that listed,
I guess you call them recognized multiple site,
static site generators.
And this is obviously excluding ones like Watermelon,
I think that was yours,
and Camel and whatever Marco called his,
all these ones that people just rolled their own,
that's not including any of them
because I think the number would be
in the tens of thousands otherwise.
But in terms of multiple site recognized
independent static site generators,
the list was over 450 and it's climbing every single week.
So there's definitely a lot of interest in this
and it is gathering momentum,
but the two most popular are Jekyll and GoHugo.
So it's a good thing you've used Jekyll
and I've used Hugo, so we can cover off those top two.
So what makes a good static site generator, do you think?
- I think that's going to depend on who you're going to ask.
I think static site generators,
these the way I understand them are mostly appealing to technical people, to
programmers and so for programmers what would make a good static website
generator is for it not to get in the way because that's what's going to
entice you to use something like that something you know static and simple
instead of something like WordPress that it doesn't get in the way that there's
just not a lot of stuff but on the other hand you don't want to like when you
you want to plug in things like you want a atom feed
or you want, I don't know, whatever.
The common things you might want to add
that you don't have to do it from scratch,
but you just plug in a couple of components.
You don't write any of your own Ruby or Go
or JavaScript code.
You just write Markdown, hit a bunch of switches,
and that's it.
- I absolutely agree.
And I think that it's the sort of thing that it's difficult
to have a static site generator that is easy
for the average person to use necessarily.
But I think different aspects of,
there are different things that make static site generators
better or worse, I think, than others.
So some are better than others.
And I guess the first and foremost reason I can think of,
you know, from a development point of view is the speed,
the speed to generate the entire site.
That's probably the biggest one.
Because the last thing you want to do is to make a change,
hit generate, and then go and get a coffee
and come back in five minutes.
I mean, that's not really what you want.
You want something that's really, really fast to do that.
And particularly, I think also the other sub requirement
for that is it has to be very fast
at rendering a local copy for local development.
So if you run it up on your local machine, you hit save,
how long does it take for that change to ripple through
to the final product,
which you can then look at in a browser, let's say.
So I think that's key.
And I also think that it's important
with these static site generators
that they're actively maintained.
And I know you could argue that
for a lot of open source stuff.
So Jekyll is open source, if I remember correctly.
Hugo is, yeah, Hugo is as well.
And that community around actively maintaining them,
that's really important because the problem with SSGs
is that it's unlike something like WordPress
where you can just like log in
And if you wanna go with the basic defaults, you can,
or you can tweak it to a point.
There's a limit to how far you can go,
but there's still a lot of people using it.
And in the case of static site generators,
obviously you need something that's actively maintained
'cause if you have got something that's actively maintained,
then it's going to get a lot of support
'cause there's a community behind it.
And that's important.
I guess the other thing I think is really important as well
is that there has to be good documentation.
One of my only, I say only,
one of my frustrations with GoHugo
is that the documentation is not necessarily always
up to date as you would like.
And that's just because as a product goes,
it's still relatively early on.
I think it's up to 0.46 or 48, something like that.
So it hasn't even hit technically 1.0
for whatever 1.0 is worth,
but it's still very early in its lifespan, I guess you could say.
It's only been around for a few years.
So, the interesting thing is that I rely more on forums
and looking at other people's examples
than I actually get from the documentation.
The documentation looks good, but sometimes it's so far out of date.
It's getting better, but in any case.
So, okay.
So, specifically with Hugo,
The thing that I found amazing when I first tried it
was that you can actually generate static files
in only a few seconds, even on a crummy Mac mini like mine.
And that's in local memory.
So you can actually run up a web server in local memory
without any other tools.
And so, 'cause normally I was used to running MAMP
and MAMP, you know, so I can run up an NGINX
or PHP environment that matches my server configuration.
But I don't need that with Hugo.
they could just be loaded up into memory.
And if you then go to localhost port 1313,
it'll just load directly from the memory
on your local machine.
And it basically waits and monitors for any file changes
on the hard drive, sorry, solid state drive,
whatever we're using these days.
I think everyone's on SSDs by now, aren't they?
So yes, spinning rust can go on and rust, please.
Anyhow, and it'll actually look for file changes
on the directory that you're executing it from.
and if it detects any,
it'll automatically regenerate the site
and then that will boot the browser
and the page will reload all pretty much automatically.
It doesn't work every single time.
Like sometimes I still have to manually refresh the webpage,
but generally speaking, it works probably 95% of the time
and it is ridiculously fast.
I mean, I can make a change
if I'm editing in Sublime Text,
which is kind of my desktop editor of choice
and I make a change to the source file,
I hit save within a second,
maybe two at the absolute outside,
it'll regenerate all of those pages
and it'll boot the browser and I'll see the result,
which is fantastic.
- I just checked with all of the Nozbe Hugo websites
because you're right, I didn't mention speed
because I was thinking of people making personal blogs
or something like that.
But when you have something big and complex,
I remember one of the, perhaps the biggest reason
why we chose Hugo was exactly that,
that everything else was too slow.
And I'm looking at the, I just hit the server.
It was pretty easy to install.
I did it just as we were explaining this
and there's thousands of pages generated in 6.9 seconds.
- Nice.
- Yeah.
- Well, the other thing that I think
is also very good with static site generators,
and I guess you could argue that Statomic is similar
because it is flat file.
and this is more of an advantage of flat file,
but I think all static site generators
are flat file by nature.
But when you're in control of the site,
you're able to use, if you like Git, for example,
you can use a Git workflow for publishing.
And that's also quite good.
If you've got a lot of, I suppose,
call them non-tech savvy people or users for the website,
then you're probably gonna end up needing
some kind of a CMS, I guess.
That's the only downside.
There are sort of ways and means
you can sort of get around that.
So there are some, you can actually create a,
and there are editing solutions
for editing like a file directory.
You could potentially say,
hey, here's how you create a new webpage
and just show them how to use a basic tool
for going to an FTP to a website.
And you sort of like,
the way I've done it with Bubble Sort, for example,
is that I've loaded in a,
it's a hidden URL specifically with PHP
and then you log in with credentials and such
to allow you to directly edit files
in the actual content directory.
So it's possible for other people to modify those.
But generally speaking,
even that requires some level of knowledge.
It's not as pretty as a CMS.
It does work, but you know, it's sort of like, hmm.
So in any case, I guess the final advantage,
want to mention quickly about static sites, whether they're statically generated or just
static static static, which is not an expression but still, is security. And because there's
no active scripts executing, there's no exploits really, well technically there's hardly any
exploits, certainly none from the website generation itself. So it's about as safe as
as you can pretty much get
from those sorts of vulnerabilities.
And there's no PHP or patching needed
for anything like that,
which is something you always got to stay on top of
if you've got a dynamic site.
- I was looking up Statamik earlier
because I know that's what you are using.
And on their website,
they're quoting someone saying that,
like a third or something like that
of like website hacks or something like that
happens on WordPress sites.
And that's not surprising because when you have,
like WordPress is a pretty good piece of software,
but it's a really big piece of software.
And there's always bugs and stuff like that.
And there's bugs in PHP that are found too.
And when you have something like that running publicly,
then it's different when you, you know,
when you have someone whose job it is
to keep updating it and just maintain it.
But most of the time you just want the website.
Like you don't want a programmer or technical person
to keep overseeing that you have absolutely everything
most up to date.
And whenever there's like a serious vulnerability
discovered that you patch it like, you know,
as fast as possible.
With static websites,
you don't have to worry about something like that.
And I wanted to add one more just piece of data
about the speed of Hugo.
I checked and the less than seven seconds measured
on the nozbe.com websites, that's for 23,000 files.
- Wow.
- Yeah.
- That's impressive.
- Most of it is like static files
that are duplicated for multiple languages,
but still, that's 23,000 files,
thousands of which are actual pages in multiple languages.
And that happens in seven seconds, which is not a lot.
- That's pretty, it's pretty impressive.
And comparatives with, you know, like Jekyll, for example,
is that, you know, that could be anywhere from a quarter
to a 10th of the amount of time
to render the same thing in with Jekyll.
But then the thing that, I guess,
when I was reading up on Hugo,
and it's funny actually that I was trying to figure out
who it was or which tweet that I saw that mentioned Hugo.
It was actually one of yours you'd mentioned
that you guys were playing with it.
And I sort of started digging into it
and a few other people on Twitter had mentioned this as recently as like two years ago
and it took me a while to give it a shot, but when I did, I was actually stunned at how much faster it was
and I think it is the fastest currently, the fastest static site generator that there is
it is pretty amazing actually how fast it is, but anyhow
so one of the other things that I just want to mention regarding static site generators
and static sites in general is that you can't just say
across the board like I am going to go completely static
because sometimes that's not always the best answer. There are always going to be
well not always but I think that there are there's definitely scope for a
hybrid
solution where you do rely on
it's a static site yes but you either have client-side execution or
you can leverage existing third-party stuff if you want to
So for example, when I moved Bubble Sort to GoHugo,
I didn't want the show email account exposed to,
you know, bots that are gonna crawl around
and then start spamming the crap out
of your poor email accounts.
That's not what you want,
which is a mistake I made early on, but nevermind that.
So anyway, there's a web service that I use
called 99Inbound for the form posting, and it works fine.
So other than that, technically it's 100% static,
but there is that one element.
So if you need to have some kind of dynamic elements,
you can look at doing something in the client,
in the browser,
or you can do something with a third party services.
I guess the other thing is you could also potentially,
if you want to,
there's no reason why you couldn't enable PHP
for a set of files or a series of files,
rather than just making index PHP,
the file that everything hits every single time.
And that could be still useful to have
some kind of a hybrid solution,
depending upon what you're trying to achieve.
but I don't pretend that static site generators
are the answer to absolutely every problem.
But if you're trying to reach a point
where you've got a high performing site
that can work on a low end server,
so be economical and basically can be,
your development can be extremely fast,
then I think that static site generators
are definitely worth serious consideration.
- Yeah, I agree.
If you have some technical skills
to be able to deal with a tool like Jekyll
for Hugo, then for a vast majority of use cases
for a website, it's enough.
Like you probably won't need dynamic stuff
that you can't solve with JavaScript
and it will save you a lot of pain.
But I do think in the future,
the approach of something like StataMaker,
like some sort of hybrid approach
might be really interesting.
Like you said yourself that you hit
limits of Statamix, the product as it is today, but an approach like this might be interesting
in the future, where it's mostly statically generated and something fast, like Hugo's
written in Go, so that's quite fast, but then you do have a CMS-like interface to be able
to edit those things, so that it's more accessible to people who are not as technically inclined,
And also so that you can plug in bits of dynamic behavior
that's difficult to achieve otherwise.
But still mostly rely on static generation
so that you get something that's really fast
and doesn't get in the way.
- Yeah, I think you got a point there.
And also compute power and especially,
well, parallel compute power anyhow.
It's not going up as fast as it used to,
but it is still accelerating year over year.
And for $5 a month now at Linode, for example,
you can get a $5 US a month,
you can get a reasonable entry-level VPS
that is quite capable and can handle running Statomik
at a moderate sort of scale.
And in another two or three years time,
that same $5 is gonna get you something
that's more powerful again.
So perhaps the whole execution problem
and issues with caching get resolved to a point
maybe it's because currently it's built on PHP so StaticMix built on PHP and it's Laravel is the
is what they're using as the basis for it essentially a whole bunch of other plugins.
The point is that it's quite possible that if that's rewritten and done on a higher performing
you know language like Go then yes it's quite possible that that is the right answer in the end
But for the moment at least, I'm... I guess you could say I'm in the honeymoon period with Hugo.
And it's... yeah, I'm sure there's going to have issues with her further along.
But for the moment at least, it's doing what I need.
And the only other thing that I wanted to mention as well about just about caching
is that a few years ago, I came across something called Varnish.
And have you ever used Varnish at all?
No, I did not.
Okay, so Varnish is an open source,
essentially it's a reverse proxy
that handles caching between your web server,
let's say it's Nginx or Apache, for example.
And it was actually released in 2006
and a lot of people swore by it,
they'd use that as their caching,
essentially, like I say, it's acting as a reverse proxy
and it's sort of, it's a one size fits all kind
of a solution and it can be tailored quite specifically
to different parts of a site.
And I actually got how to play with that.
And I also had, because I prefer Nginx over Apache,
I have done for years.
And so I was using Nginx and the FastCGI cache
actually performs very similarly these days to Varnish.
So when it came to the caching on,
'cause I still do, you still use the Nginx caching
for the static sites.
And it's just, it was a bit of a wash really.
So I kind of gave up on Varnish
'cause Varnish was another necessary complexity
at that point.
It wasn't giving me any additional performance.
So I just wanted to throw that in there
because a lot of people sort of swore by Varnish
and said, "Oh yeah, you gotta use Varnish and so on.
"It's the best and everything."
But I think that the other options that are out there
that are built in, 'cause Nginx, obviously,
it's just, if I added Varnish on top of Nginx,
then I've gotta maintain both Nginx as well as Varnish.
It takes memory, blah, blah, blah.
So you're just better off, I think, anyway,
just sticking with the Nginx caching.
So that's been my experience anyhow.
And that applies to static or dynamic sites.
But I also had issues with caching as it related to,
like I said, two levels of caching.
So with Statomic, so you had the caching in Statomic,
then you had the caching in NGINX.
And then when I was using Varnish,
you only had caching in Varnish.
So you had three levels of caching.
And it was, yeah, it would have some bizarre behavior
where sometimes you had to seriously bust
every single one of them in order for it to actually show,
push a change out, which was very frustrating.
Whereas with Hugo, you just statically generate it
and the nginx cache is the only cache and that's it.
And I haven't had any issues with it at all.
And what I do with Hugo is that I have a cron job
that executes every five minutes
and it just regenerates the site.
So if I make changes to the source files,
every five minutes, it just re crunches those
and it automatically syncs up and publishes that
and it goes live.
So I can actually schedule a file to go live
at a certain time and date.
You can also do like conditional stuff
like most of these, you know, static site generators.
You can actually specify a time to publish
and a time to unpublish if you want to.
So if you want something on the site that's time limited,
you can do that as well.
And as time moves forward,
it just automatically generates the site
and it displays what you would expect it to display,
which is really great.
So it kind of looks like a bit of a dynamic site,
but in truth, it's not.
So I think that's fantastic.
I'm not surprised at all to hear your caching problems.
There's a saying or a joke, I guess, among programmers
that the three most difficult problems in computing
are off by one errors, naming things,
and cache invalidation.
And when you have to use a cache,
that often indicates a bigger problem
that some piece of computation that should be fast is not.
And if you can, if it's practical to just avoid
the problem altogether by skipping the computation
or changing it to something that's just really fast
and doesn't need a cache, you're going to save yourself
a ton of headache over the long term.
And that's like, we're talking about, you know,
static site generators or any other thing in computing.
You just time and time and again,
you get really bitten by caches all the time.
- Awesome.
Alrighty, well, if you wanna talk more about this,
you can reach me on mastodon@chijji@engineered.space,
or you can follow @engineered_net on Twitter
to see show-related announcements.
If you're enjoying Pragmatic
and you wanna support the show, you can,
like some of our backers, Karsten Hansen and John Whitlow.
They and many others are patrons of the show via Patreon,
and you can find it at patreon.com/johnchijji,
or one word.
Patreon rewards include a named thank you on the website,
a named thank you at the end of episodes,
access to pages of raw show notes,
as well as ad-free high quality releases of every episode.
So if you'd like to contribute something, anything at all,
there's lots of great rewards and beyond that,
it's all very much appreciated.
If you're not in a position to support the show
via Patreon, that's fine.
You can still help by leaving a rating in iTunes,
favoriting the episode,
or the show in your podcast player app of choice,
where it's supported,
or by sharing the episode or show on social media.
It helps other people find out about the show
and that's all very much appreciated as well.
Pragmatic is part of the Engineer Network
and you can find it at engineer.network
along with other great shows like Causality,
which is a solo podcast I do
that looks at cause and effect of major events
and disasters in history,
including Three Mile Island,
the Challenger Space Shuttle,
and recently Chernobyl and lots more.
Causality is on track to overtake Pragmatic
as it grows in popularity.
So if you haven't tried it out yet, be sure you do.
Now, if you'd like to get in touch with Radek,
what's the best way for people
to get in touch with you, mate?
So you can find me on Twitter @radxp, R-A-D-E-X-P.
You can listen to me and my friend Michael
every week on the podcast, thepodcast.fm.
And I blog about programming at radx.io.
- Fantastic, all right, cool.
Well, a special thank you again to our patrons
and a big thank you to everyone for listening
and thank you for coming back on the show, Radek.
It's been great.
- Thank you so much for inviting me again.
And I wanted to also really recommend Causality.
I really dig that show.
- Oh, cool.
- Thank you very much.
(upbeat music)
[Music]
(dramatic music)