Pragmatic 62: Send It Into The Void Baby

I've, my wife just bought me some, some cappuccino mugs now that fit under the spout properly. So I've actually made myself a cappuccino rather than a latte, which is very subtle, but still close enough, I guess. And yeah, so I'm buzzing, I'm wired. If my fingertips are tingling, is that mean I've had too much coffee? Anyhow. That might mean you've had too much coffee. I think I crossed the line. Welcome to Pragmatic. Pragmatic is a weekly discussion show contemplating the practical application of technology. Exploring the real-world trade-offs, we look at how great ideas are transformed into products and services that can change our lives. Nothing is as simple as it seems. This episode is sponsored by Lynda.com. Lynda.com is the easy and affordable way to learn where you can instantly stream thousands of courses created by experts in their fields of business, software, web development, graphic design and lots and lots more. Maybe you want to master Excel, learn negotiation tactics, build a website or tweak your Photoshop skills. Visit lynda.com/pragmatic to feed your curious mind and to get a free 10-day trial. There's something for everyone. So if you've ever wanted to learn something new, what are you waiting for? This episode of Pragmatic is also sponsored by Sapient Pair and their iOS app, Shopee. Shopee is a collaborative shopping list app that's simple and easy to use, with great features like pocket lock, smart ordering, and real-time collaborative updating. Shopping lists aren't to-do lists, and Shopee doesn't just help organize your shopping list items, but it also helps you shopping from start to finish. It's free on the iOS app store, so check it out at sapientpair, that's S-A-P-I-E-N-T, dash pair as in two dot com slash pragmatic for more information. And we'll talk about those sponsors a little bit more during the show. I'm your host, John Gigi, and I'm joined today by my co-host, Vic Hudson. How's it going, Vic? It's going good, John. How are you? I'm going very well. Thank you. To today we have a short show. Yes, this is going to be a short show. The shortest. I have never prepared more for an episode. I have never had a topic that is this complicated to cover and it's actually the most highly voted most popular topic when we had the topic voting up on the site. It was the number one request for fans and seeing as how this is the second last episode of Pragmatic it seems only fair as a final thank you to the fans of the show that I cover off the most popular topic on the list. And that topic is how the internet works, which is a very broad kind of topic. It's kind of a lot to it. There's a lot of pieces. So, I'm not going to, I'm just going to dive in because let's face it, if I don't get started, I'll be here for hours. In fact, I probably will be here for hours. So, strap yourself in for a short show. Are you ready, Vic? I am ready. OK, Fentah, let's do it. Let's do it, man. OK, so I want to cover the evolution of the Internet, but I want to take it step by step. So when the blocks come together that they sort of make sense, there are a few fair warnings at the beginning. OK, there are a lot of acronyms that I'm going to be mentioning in this episode. Don't blame me, please blame every other geek in the world that invented this stuff and thought acronyms made things sound cool, okay? Because I don't know, there's just a lot of acronyms in this stuff. I don't get it, whatever. As I said, it's gonna be a short show, brace yourself. I'm not going to go down to the voltage level, okay? I'm not gonna talk about Manchester encoding and blah blah blah blah blah. I'm only interested in the key points and there are a lot of key points, but I'm trying to help form a clearer picture as to how it works conceptually. So I'm not gonna talk about plus minus 5 volts signaling, I'm not going to talk about TTL level, I'm not going to talk about any of the actual lowest possible level hardware level, this voltage, this current, you know, that's a current loop that isn't, you know, I'm not going to talk about that level because the problem is that honestly, I just don't think that there's as much value in that. That said, you know, electrical engineer with electronics background, of course, that stuff is of interest and I could talk about it, but I honestly think that, you know, let's just draw the line and say we're going to deal with the levels above that. So no lower. I'm also not going to talk about every single possible permutation combination of every standard that there ever was. I'm only covering the ones that have historical significance and I'm only covering those that have influenced where we are to where we are now and how we've got to where we are now and I think most importantly as well the ones that are currently in use. So and I'm not to do comparatives between different technologies. So hopefully by observing those rules we'll get through this before we all die. Lovely. Okay history. Got to start with history. So Leonard Kleinrock at MIT, July 1961, he published a paper on packet switching and later a book on the same subject that was in 1964. Now up until that time data between computers was transferred pretty much point to point and it wasn't actually transferred it was transferred as a serial data stream. Now the thing as is the case of a lot of inventions yeah people say the Wright brothers invented flying and all that sort of stuff and yeah whether that's true or not I guess I suppose it is true but kind of but there was a bunch of other people working on the same time as as was the case with packet data and packet switching concepts and packet switch networks. So MIT, RAND, NPL, they were all developing very similar packet networking concepts in parallel during the 60s. But the name packet was actually coined from NPL's work. Now, a guy called, sorry, I shouldn't say guy, it's a group of people. So you got Bolt, Berenek and Newman, BBN, it's the name of the company, they won a tender in 1968 to build the first ever interface message processor, or what we would come to refer to as an Ethernet switch. So 1968, and the actual first models of that were used in something called ARPANET. I always seem to say that wrong, ARPANET, anyway. And that was actually developed and implemented at UCLA in late 1969. So that was really the first recognized computer network using packet switching and what would become Ethernet switches. Now there's a long long history of ARPANET, the people involved. I don't want to go into all of the details but some key points along the way. So NCP, Network Communications Protocol, that was the predominant networking standard on on the early ARPANET and it managed essentially how the packets were handled and directed through the network. So, NCP, that was the... so there is... we haven't got to TCP yet, okay? So, a guy called Ray Tomlinson wrote the first program to read and send what they originally... the term email wasn't coined until a few years later but electronic mails that was in 1972 and that was using the ARPANET. So it was first defined TCP to the transmission control protocol was first defined in 1974 and by 1978 the first form of internet protocol or IP as you know we've come to call it was completed the resulting standard was called TCP/IP which is the standard that the internet is essentially built on. Now, the thing is that despite the fact those standards were in place by the late 70s, well, so I should say the standards were written, drafted and existed. Implementations took a few more years to sort of trickle out. So it wasn't actually until 1983 that the ARPANET had actually switched their entire system to TCP/IP. And on the ARPANET at least, NCP had died. There were still other fledgling networks around and so on and so forth, but these were all isolated networks essentially. They started to become more cross-connected during the 80s. Now I'm not going to go into every little detail of all of these, I just note them for their historical significance. So 1981 CSNET, 1982 SMTP, 1983 DNS, then we skip forward to 1990 for Archie, 1991 World Wide Web, 1993 the Mosaic Web Browser. some key points along the way and yes, don't worry, I'm going to go into a lot of these later on. But I thought it'd be interesting before we jump back down to the low level, just to quickly cover off some of the internet-based services history. And what people don't realize is that, you know, because you think about ARPANET in the late 70s, you think about the 80s and the proliferation of TCP/IP, but it was pretty raw stuff. There was IP address, there was like DNS, although DNS came around in the early days, 83, it was, you know, implementations of DNS and the widespread usage of it and internet services built on all of this stuff really didn't actually become a thing. Like I said, I mean, Archie and the World Wide Web 1990, 1991. That's when you start to see some of the very basic internet what we would consider an internet service, you know, started around then. So, okay, IMDB, the Internet Movie Database, 1990. It wasn't until 1995 that we got Craigslist, we got Amazon and we got eBay. So 1995. So that's quite a gap. That's a considerable gap when you think about it. If you want to consider ARPANET switching to TCP/IP, compare that to when you got Amazon. Yeah, what's that? 13 years? Hang on. Yeah, 13 years. That's quite a gap. So 1996, Hotmail. Still have my Hotmail account. Oh, yeah. Yeah, I had one of those. You had one. Did you let it go? Yeah. My wife still has hers, too. So it's now owned by Microsoft. It became Windows Live and now it's become Outlook, you know, because, well, that's just what Microsoft does. you know, buy something and then brand it under the Windows/Office moniker, right? That's just what they do. All right. So, so that was Hotmail '96. 1998, PayPal. OK. And then, of course, the big one, Google Search. Google. Yes. Google. Google. So, yeah, 1998 was Google time. And that's when everything really started to accelerate. Blame Google. 2001, we see our bestest friends, Wikipedia, well often, well linked and some people would say overlinked in my show notes, but stiff. 2003, got a whole bunch of really good ones then. Skype, which we're using right now. MySpace, which I think is still alive, but mostly dead. LinkedIn, which is regrettably still alive. iTunes. Yeah, no kidding. The iTunes store. Oh yeah. And of course a crowd favorite, the Pirate Bay 2003. Exactly. 2004, Facebook, or as I prefer to call it fast book anyway, whatever. 2005, YouTube, which apparently makes no money. So I heard recently, whatever. 2006 was Twitter and 2008, we get Dropbox and Spotify. So there's been a lot of other services I didn't mention, but I thought it was interesting to think about the services that we use today and just how far back they go in terms of history. And the funny thing for me is that I use Google search every day. A lot of people will say, "Oh yeah, I use DuckDuckGo and I use..." I don't know, does anyone use Bing? I guess someone does. Anyway, so all these different people, you know, majority of people still use Google search. That's actually been around for a very long time. A lot of people buy stuff on Amazon. So that's more of a North American thing. People outside North America aren't so big into Amazon simply because Amazon don't have the, you know, all the benefits of the Amazon Prime membership and all that sort of stuff doesn't have much benefits in the outside of the US. But you know, even so it's still been around now for 20 years. That's pretty impressive. eBay, well, I think we all tolerate eBay. Sometimes I still sell stuff on eBay. And once I do that, I swear I'll never use it again. And then I come back two years later and use it again to sell something. So I don't know, I guess I never learned, but whatever. And IMDB, yeah, I use that sometimes. So, but anyway, interesting though, just how far back some of this stuff does and doesn't go. But all this stuff is built on the internet. Without the internet, it wouldn't exist. So that's what can be built on the internet. A little bit of the early history. So now, let's talk about how it actually does work. So, hopefully appetites are wetted at this point. at this point. Let's get connected. Let's get connected. Oh my god, yes, let's do that. Okay, so let's start with the beginning, which is a point-to-point data connection. So, the concept behind a point-to-point data connection is mind-bendingly simple. It's just a conversation between two people or two devices. I'm going to raise my hand and say, "Are you ready to get some information?" And the other person is going to say, "Yes." I'm going to raise my hand and say, "Yes, I'm ready." Once I see them raise their hand, I'm going to send a bunch of that information continuously and then I'm going to give them another signal to say, "Hey, I'm done." You know, I'm going to drop my hand or whatever I'm going to do. And then they're going to say they got it once they got it by nodding or dropping their hand and saying, "Yes, I got it." And that's pretty much it. A little bit of handshaking, a burst of data, and that's the end. Point-to-point serial data communications. So, not the difference between serial and parallel, just note, not talking about that. serial and parallel. Serial data is just that the data is sent consecutively with a common spacing and sometimes you'll have a clock that you then, you know, sometimes it's self-recovering clock in high-speed serial or it's a parallel clock. Either way, you know, you clock the data off and you extract it that way. Whereas parallel, you'll send a whole bunch of bits all in parallel with a common clock which has all sorts of other problems and I've talked about that previously. So, no, No, I'm not talking about the difference between serial and parallel. I'm talking about the basic idea of handshake. Here's a continuous stream of data and I'll tell you when it begins, I'll tell you when it ends and you will extract it and then go and do whatever you're gonna do with it. So from a conceptual point of view, that is a basic serial point-to-point data communication link. Very few layers to it, not much going on there. And that's the way it all began. one of the evolutions along the way before we progressed to the networking idea and packets and everything was an idea of a token ring. Have you ever heard of a token ring system? - I've heard the term. - Yeah, token rings are something that I've come across in automation systems. And there's a whole bunch of reasons why token rings are good. And there's even more reasons why they're bad, but anyway. So like I said, in process automation, I come across them. Some of them are actual genuine token rings. The other ones are hybrids with a master-slave idea. But anyway, bottom line is it comes back to guaranteed delivery time and time slots and durations and so on. So, companies like Siemens have used this system, but they also have their own system called Profinet that sits over the top of Industrial Ethernet, which is a derivative of the Ethernet standard. But I'm talking more about Profibus. And Profibus is sort of a hybrid token ring system. And the idea is, think about it like you got a bunch devices all connected together, but each device is connected to the next device in a circle. And each of those connections is a point to point link, just like we said before. Okay. But yeah, but think of it like the token is like a baton or a talking stick, if you'd like. You know, you know, when people argue in a big group, and they say, you don't have the talking stick right now, you know, that's my talking stick. Have you ever heard that one? I mean, I have. Yeah, it's just terrible that we've heard of that. But yeah, okay. So yeah, you can't talk right now. I got the talking stick. So it's kind of like that. So you pass the talking stick. I thought we were going to talk about the Tolkien ring. That's the one ring to rule them all. This is the one ring that no one wanted. Anyhow, so yeah, as a this one doesn't make you disappear. It just slows you down. So anyhow, okay, Tolkien So device one says here have a token and it's like, okay, I have the token now and no one else has got it. No one else is using it. So I'm going to put some data on it. Device one, I want to talk to device three, but to get there, I've got to go to device two first. Okay. So I put that associate that data and I pass that on to device two. Device two has a look at the data with the token. It says, oh, this is not for me. I shall pass it on to the next bloke. Pass it on to the next one and the next one opens it. It's device three. Oh, great. I'm the intended recipient and it pulls out the data and does what it's going to do with it And it may have a response for device one to say I really didn't like your data or whatever it does who cares point is That's the way a token ring works. You get one person a hardware linked list then Yeah, sort of kind of kind of sort of yeah. Yeah, it's it's the idea is that It's the ultimate in data control. You got one person at a time and and one person, one device at a time and the data gets passed around. And there's all sorts of permutations and improvements to the token ring system. But honestly, the implementation really, it can get more complicated, but it ultimately died because Ethernet, in some senses it's more complicated, but Ethernet in many respects is simpler. And kind of it's the ubiquity because of Ethernet's inevitable lower cost, one out. 'Cause if I recall correctly, Token Ring was driven primarily by IBM and Ethernet was a bit more open. I think that may have been part of it as well, but ultimately, packet switching won. Token Ring still used a lot of automation because it will give you a guaranteed delivery time, which is very important in process automation. You know that you're always gonna get a response 'cause the token can only sit with you for a certain amount of time before you got to pass it on. So that means you've got a guaranteed delivery time, whereas Ethernet, that is not the case. So it's, yeah. So that means that you essentially have a bunch of IO out in the field, and you need to know that that input, you're going to get data from that, you know, that bank of IO out in the field, you're going to get that in the next 50 milliseconds. Well, that's important that you get that message in that timeframe. So, you know, it's important in automation, but it's not so important in computer networks where you can have random variable lag, piece together the packets and re-sequence them and reconstruct the data and you're all good. So, you know, it's not, yeah, anyway. So ultimately, Ethernet won out over Token Ring. So Token Ring still exists, but it's very, very, well, it's just not popular. So anyway. Okay. So, before we talk about Ethernet networks and so on and so forth, I think it's helpful to talk about another point-to-point link. And that is the, essentially, okay. In the beginning, computers, personal computers in particular, let's focus on them for a second. Personal computers didn't have a network card because what would you need ethernet for? Yeah, even though when VIC-20 existed, networks existed and the original coaxial networks like 10BASE-5 and so on, all those things existed. But networking computers was, oh, why would you do that? It was something that only universities would do that had not lots of data to share. And home computers had, you know, there was no internet really to connect to. You could dial in, perhaps, I guess modems did exist, but they were point to point links. So, yeah, you had BBSs and all these other bits and bobs, which I'm not really going to talk about. But the point is that, you know, computers had no network interface cards. The way you would connect to anything, you know, on the Internet, when the Internet came about, really, that was through a modem. and modems would connect to your device, your computer, usually via serial port. Usually it was RS-232. Eventually that, you know, there was some that connect through parallel ports, but generally speaking, eventually that became USB. But by the time that was happening, everything was moving towards Ethernet and, you know, you know, getting towards Wi-Fi routers and, you know, to where we are today. So in the beginning, though, you know, like I said, my, the first computer in our household of VIC-20, it had no networking capability at all. and our first PC had a dial-up modem and it was a 19.2k dial-up modem Oh that was blazing! Oh it was blazing fast, oh yeah! That was using compression man! Anyhow, so yeah, a lot of early companies and users just didn't see the internet as being useful useful. It seems crazy now, but thinking back, everyone's like said, "Shrug internet, what's the big deal?" And think about it. Back then, in this, I'm talking about the late 80s, early 90s. So you'd buy your Compaq or your Gateway or your whatever the hell, and you'd have a modem, you could dial into the internet, but there wasn't much out there. What is there out there on the internet? You had horrible search engines that didn't find very much because there wasn't much to find. The majority of people using them were research and universities. The average person with a personal computer at home had no real use for it. So it didn't become all that useful. And I think that a lot of the problems were that the data rates started out being very slow. OK, coming back to the problem with this conversation is that it's hard to know where to start. Yeah. So imagine you buy yourself a computer, you've got to use a dial up modem, how do you connect, you need an internet service provider. And in that service providers, you know, let's let's go back to America online, for example, and there were hundreds of 1000s of ISPs and you would dial into them, they give you a phone number, you would dial in, and you would connect after the modems would negotiate. And they would agree on a speed and then you would be connected to their network and there the ISPs location was then connected through switches to the internet. We'll talk about how later. The point is that that dial-up connection was done over a phone line and that's because that's all we had. There were no mobile phones. ADSL hadn't really been invented at that point. I mean I think theoretically perhaps but you know the whole asynchronous digital subscriber line concepts and the filtering and everything none of that existed. It was just an analog phone line through an analog telephone exchange. God knows how many telephone exchanges you pass through between your house and the internet service provider could be on the other side of town. You just don't know. And there could be three telephone exchanges between you and they're all analog. So that was what you're up against. So the amount of noise, analog noise on those phone lines is terrible, just terrible. So that's why these modems started out at really low bit rates because they had to operate in high noise environments. So you started out with 1200, then it was 2400 baud, bits per second. Then things started to accelerate, you got 9600 baud, let's put a point in there, we're going to start pulling 9.6 kilobits per second, kilobauds, oh yeah, hour talk and like I said, our first one was 19.2k. I mean, I had a TNC modem, you know, for amateur radio that was 1200 boards. So yeah, but that's another story. I'm not talking about that because that's packet radio and no one did packet radio or hardly anyone did comparatively speaking. So let's not worry about that. So what changed though, is that some of the back end improved more information from more people saw them seeing the benefit of the internet was added all around the world and suddenly it became a more interesting place to be. So more demand meant that there needed to be faster ways to connect, to get more of this data. And that led to more ideas. Well, there was a proliferation at that time for a bunch of other efficiency reasons, away from analog exchanges to digital exchanges, ones where all the circuit switching was done digitally. And it started out in the back end, the high density, you know, switching between exchanges. And then eventually it moved its way out into the suburbs so that your suburban telephone exchanges were essentially just a an analog line from the nearest telephone exchange to your home. And that was it everything else was digital. So as soon as it came into the telephone exchange, it was digitized, and away you go. So that meant it was then possible to start mounting essentially internet specific equipment at the actual telephone exchanges. And that allowed that allowed the in it service providers to say, Okay, well, now we can cut out all the switching loss and all this other analog to digital losses, we can actually just use this lot, but the soak, they call it the last mile, right? It's not, it's not, it's not a mile, you know, last kilometer, if you want to call it a last kilometer, whatever, you know, it what it means is it's the distance from the telephone exchange to your house and whatever that distance is. It could be 100 feet, it could be 10,000 feet, hopefully not 10,000 feet, that's a long way, but it could be a long way. And the point was that because you're only dealing with that last bit, it cuts down the amount of noise that the phone line can pick up significantly and it allows you to use ADSL. So we start out with ADSL 1, then that progressed to a new standard ADSL 2 and then ADSL 2+. And and says, "Oh, 80 cell 2 plus, yeah, it's fantastic. You can get tens megabits per second." And it's like, yeah, that's great, provided you're living next to the damn telephone exchange. Soon as you're more than a few hundred feet away, here, good luck. So, and the problem with that is of course, that there's only so much data you can push down a twisted pair cable over a significant distance. And the reason why is straightforward enough. And that is that digital information on, off, all right? voltage goes on, voltage goes off, current goes on, current goes off. All of those switching transients are governed by inductance and capacitance in the cable. The larger the inductance, the larger the capacitance, the more difficult it is for those pulses to actually maintain their shape by the time they get to the other end. So the further you go down, the more capacitance inductive effect. Therefore, the more rounded the edges become, the more difficult it is to detect whether or not that is actually a one or a zero. And that is a gross oversimplification. I'm excluding quadrature amplitude modulation. I'm talking about I'm not even talking about QPSK. Oh my goodness, all that other stuff, different modulation methods, and so on and so forth. But that's the basic idea. Capacitance inductance is your enemy, because the transmission lines that were laid down were never intended for digital information. They're intended for an analog phone call, and they do that job just fine. So we're going to put ADSL hardware out there in the telephone exchanges. And then people can connect with an ADSL modem instead of a dial up modem. And that means that you're always connected, because there's nothing to dial in to connect to you're always connected. So you're not dialing in through IntelliVent exchange anymore. Technically, you're going to essentially you're superimposing digital information over the top of your analog phone line, it only goes to the exchange. And then that digital information stripped off and sent digitally through the backbones back to the ISP. So you're cutting out all that noise. But at the same time, there's still limitations how fast you can go. So obviously there's also cable modems and nowadays we have fiber optics and fiber optics is digital all the way but most people do not have fiber optic into their homes, you know, globally speaking. Certainly not now, certainly not in my country and frankly, certainly not even in the United States. It's becoming more common but I mean I don't know what the statistics are but I guarantee you the vast majority of homes do not have it. I'd be interested to know what the number is actually but... Yeah, I'd be interested to. I'd guess that most of the most of the people are connected in this country via cable modem or DSL. Yeah, exactly right. And it's the same as the case here. And some people are unfortunate enough to have to deal with satellite and all the lag that goes with that and then and the low bandwidths and all that stuff. that stuff. So the more modern approach has been to utilize the cellular networks. So where there is no analog phone connection, which you could potentially ADSL on, and there's no cable, because you know, not everywhere gets cable. You can just use a 3G network or the 4G network. Why not LTE? You know, that's pretty damn, that's screaming fast. Of course, the problem with that is it's a shared network, a shared, shared medium, which means that ADSL, dial up, cable modems, all of those things are a dedicated connection between the exchange and you. Whereas 3G and 4G is shared by anyone who can speak to that to that mobile phone tower slash sector in the cell that you're talking on. So that means that it's not just you using it. Depends on where you are. And it's down to luck. So you know, yeah, depending how I just do you feel lucky. Anyhow, I really don't do a very good cleaning. I got it. You get it. Okay, cool. All right. So during this transition, where speeds on at the client connection starts to evolve, something else happens. As the bandwidth increases, the amount of information available on the internet increases more devices because the average cost of devices in the home is reducing. So no longer is it, do you have a computer in your household in the 80s? And you'd be you know, one in 10 people maybe put their hand up? Do you have a computer in your household in the 90s? Two or three or four or five people with their hands up? Now, where are the noughts? The 2000s, as in last decade, how many people have one computer in their house? Every hand goes up? How many people have two, three or four computers in their house, more hands go up? You know I'm saying. So suddenly, it's no longer just about a point to point link. It's about, okay, I have one phone line into this house, I got one internet connection to this house, but I've got multiple devices, how do I deal with this problem? So that's when the proliferation of home private networks started to happen. And modems ceased to be just modems. And they had to evolve by implementing another functionality that existed in enterprise networks for quite some time and that's routing. Before we talk about routing, though, it's important also to just quickly mention that some operating systems supported internet connection sharing. So you can have a dial up modem connected to your computer, which would be your internet connected computer. And then what you do is you'd share your internet connection with the other computers on your network, and they would quote unquote, automatically figure it out. Some of them were less automatic than others. I'm not naming names, Windows, but I had problems and your problems became my problems and enough said. So yeah, I was really glad that internet connection sharing died and was replaced by a genuine router because let's face it, a router actually works far more reliably. So yeah, yeah, I never had good luck with the internet connection sharing. No, I don't think too many people did. It sort of kind of occasionally worked. But you know, it was a pain in the neck. And of course, the problem was you had to leave that computer on all the time. So if you wanted other net computers in the network, then your internet connected sharing computer had to be on and awake, it couldn't go to sleep and all that sort of rubbish. It was a pain in the neck. So thankfully, modems evolved and they then started to incorporate routing functionality. And that allows multiple devices to access the same internet connection because the modem is always on. So by incorporating the router into the modem, all of the computers then connect to the router slash modem. And that gives them all access to the same internet connection. So they then share that internet connection. So everyone's happy. Well, until of course you have, you know, kids, and then the kids suck all the bandwidth and then you're left with nothing. You can't even have a Scott phone call. And that's such a first world problem, but anyhow. Okay. With me so far. I'm here. You're here. Oh, okay. Does that imply that you're with me so far? - Yes. - Fantastic. Okay. Aye, aye, aye. So scratching the surface here. So much to cover. Okay. So, do, do, do, do, do, do, do. ISPs essentially then provide you that point-to-point connection into your house. And then once it's in your house, you have an ethernet connection of some kind. And of course, traditionally, it was a wired ethernet connection. However, more often than not these days, It's now Wi-Fi, a wireless ethernet connection. And you have your own little private network in your house and your own little private party. So once your modem router has connected to the ISPs network, they then route your packets to wherever they may need to go. However, there's one other little thing to talk about on ISPs before we change the subject. And I say change the subject, it's all the same subject, but you know what I mean? Same sub subject, is sub subject a word? Anyway, ISPs, one of their problems is they have a bank of IP addresses that they have to dole out to people that connect to their system. They have a fixed number, so they got to manage them carefully. You know, they're gonna be miserly about it to go to some kind of a rationale reason. So there's two flavors. There's the fixed IP, which everyone says they wants, but they don't probably need. And there's the dynamic IP address. It's pretty simple. Fixed IP means I'm going to log in as me, John Chigi to the ISP. And they're gonna say, Oh, hey, John, here's your IP. Same as always. Hope you're happy. Give me an extra 10 bucks a month. Yeah, pretty much. That's 10 bucks. Come on, man. It's just a fixed IP. Anyway, dynamic one says, Oh, hey, John, it's you. Yeah, because you're stingy and you're cheap, you can keep 10 bucks a month but you ain't getting that same IP address. What's that? It's still the same as last week? Whatever. Maybe it won't be but maybe it will be. Anyway, so they just get a big bucket and they just pick whatever number they want out of the bucket. So that reuse of IP addresses, that the dynamic method meant that the ISPs can allocate whatever they like to you from a pool and it's just easier and it's more efficient, theoretically, and perhaps it's more of a tradition thing. Honestly, I feel like the days of dynamic IP addresses are kind of over to an extent, because if you think about it, it makes sense when you've got a lot of people connecting and disconnecting all the time, but with always on internet connections, so my ADSL is always connected, unless of course, you know, power goes out, or the modem dies, or someone pulls out the cable from the back of the modem or, you know, insert reason here that is not a common occurrence. Generally speaking, I connect to my ISP and I'm going to have the same connection up for days, weeks, maybe even months. Yeah. Now they can't reach max capacity, it's going to become a de facto fixed IP anyway. Exactly. So my pool, as more as a larger percentage or proportion, I'm going to think about of my dynamic pool become always on connections, it ceases to become a dynamic pool that can be reused. All it becomes is a highly defragmented pool of IP addresses of which there's very little point in reusing them. When you had dial-ups, a dial-up modem, it made absolutely great sense. Or when you had a handful of people, not a handful, but a relatively small proportion of people with always on internet connections, I think it made sense. But I think it just makes less and less sense these days. And I really don't know where they get off charging 10 bucks a month extra for a fixed IP address. So even if you had a dynamic IP address and just by the way for the record I have a dynamic IP address because I'm stingy. No actually although maybe I am stingy but the point is I don't need one and I know I don't need one. So for a very long time there were a bunch of services that were around that could get help you around this problem. Maybe you've heard of one of the one or two of them. I'm going to mention the one I used to use. That was DYN DNS. Heard of them? I think so. Yeah. Okay. DYN DNS short for dynamic DNS. Yeah, I used those. I used them. Yeah. So I thought that was who it was. Yes. So DYN... I ran a hobby web server several years back. And that's what I used to negotiate. Cool. Yes. So I used DYN DNS for quite a while. And it was a free service for quite a while. But like so many free services, they realized, "Hey, we could make money from this and it seems to be a free service." I mean, I could always pay a nominal amount, but I stopped needing it and I stopped bothering with it. And there's a whole bunch of reasons why, which we will get to later on in the episode, don't worry. But for the moment, let's assume that you have to have a publicly known, externally available IP address in order for you to essentially connect back into your home system when you're out and about. That would be a use for an IP address. If you're running a web server off your home computer, which is highly inadvisable, but you know, you can, if you really want to, then you would need a fixed, a static IP address. But if you don't want to pay for one, what D1N DNS did is you would point your modem, modem router, to the D1N DNS service. You would give it your login credentials. and what would happen is every time you logged in to your ISP, it would send a message to DUI and DNS saying, "Hey, here's the IP address that my ISP has assigned to me." And then DUI and DNS would give you, essentially you could then connect that to a domain and you could say, "Right, I'm going to go to this domain." And it will simply redirect you to the IP address that was dynamically allocated to you by your ISP. Hey, Presto, I have a quote unquote, "fixed-ish" kind of sort of fixed-ish IP address. So that was one of the ways around it. And there are other ways, but that was one way. And, uh, yeah, so if you wanted to get, you know, access to your, you know, laptop from home using, you know, Back to My Mac or different VNC software, like maybe like Screens VNC or Teleport or there's a whole bunch of different ones out there. There's two examples. There's plenty, you know, Cy-, oh, not Cyberduck, God, that's FTP. Ah, real VNC. Hmm, tight VNC, loose VNC, whatever. Anyway, yeah, that's that. So there you go. But the bottom line is, the downside though, is you're relying on a third party service to provide you that access. And then eventually DOI and DNS got, got, did they got wise? Got smart? I don't know. Got greedy? Nah, probably not. They have every right to earn money, to sustain their existence. And that's fine. It's just that I didn't need the service that much because I found another way. found another way which we'll talk about later anyway. It solved the dynamic problem as it was so there you go. So that's a bit about ISPs and that last mile connection not going to go into too much more detail about that and the reason I'm not is because there is so much more still to talk about. But before we do I'd like to talk about our first sponsor and that is Lynda.com. Now Lynda.com is for problem solvers for curious and people who want to make things happen. You can instantly stream thousands of courses created by experts in their fields of business, software, web development, graphic design, audio and lots and lots more. Wait a minute, a list here. Now they have an enormous library, you might even say gargantuan of titles that you can choose from with new courses added each and every day and that makes sure that their library is relevant and up-to-date. Now, lin.com is used by millions of people around the world, there's no exaggeration and has over 3,000 courses on topics like web development, photography, visual design, business and software training as well like Excel, WordPress, Photoshop. They work directly with experts from many different industries and software development companies themselves and that allows them to provide you with timely training. Often the same day the newest release becomes available so that you know you have the latest information the moment you're most likely to need it. Now, they're tutorials and nothing at all like the homemade ones you might find on YouTube. And I found those that sometimes if you're lucky, maybe if you're lucky, you might find a little snippet buried unindexed somewhere in the middle of the YouTube clip that tells you what you actually need to know. But lynda.com make high quality, easy to follow and well indexed. See, that's the big thing for me is the indexing. So you can find out exactly that piece of information you wanna know it's indexed. And their video tutorials have transcripts. They're broken down into easily searchable sections, and the bite-sized piece approach makes it very easy to stop and pick up wherever you left off, whenever you need to, so you can learn at your own pace, in your own way, and in your own time. Now, whether you're a complete beginner with no knowledge at all about the subject in question, or you've been a moderate to an advanced user looking to brush up on the latest version of whatever software you're using, lynda.com has courses that span that entire range of experience. You can learn on the go as well, Since lynda.com has iPhone, iPad, and Android apps, they also support playlists. They provide certificates as well as evidence when you complete courses. And if you're on LinkedIn, you can publish them directly to your profile. Many, many years ago, I left Windows behind as much as I could anyhow, 'cause I still use it at work, but nevermind that. And I switched to a Mac and I've been using one ever since. But I got stuck into lynda.com's Tiger the Basics tutorial. I followed up then with Leopard's new features and essential training the next year. but that was getting close to a decade ago now. Now, linda.com, therefore they've been around a very long time. They're not a new thing, and they've been around for so long because they are so good. Now, some interesting courses available right now include Excel 2013 Power Shortcuts, always handy to know. And if you're a regular listener, show you know how much I love Excel. Another interesting automation related course called Up and Running with If This Then That, if you're into IFTTT. Now, there's also courses on WordPress, Photoshop, Google Drive, Google sites, SEO fundamentals, if that's your sort of thing, there really is something for everyone. Now, a lynda.com membership will give you unlimited access to all of that stuff on hundreds of topics for one flat rate, and that's great. One flat rate, whether you're looking into becoming an industry expert, or if you're just passionate about a hobby, or you wanna learn something new, you can visit lynda.com/pragmatic and sign up for a free 10-day trial. It's free to try, and once you do, you'll see exactly what I'm on about why I think it's so good. Thank you once again to Lynda.com for sponsoring Pragmatic. Okay, next, let's talk about the stack. This is when you say what stack? It's getting tall. I'm talking about the stack. Pancakes. The pancake stack. All right, let's go back to the read and restart. No, we're not restarting it. No, no, no, no, no, no. That was a terribly bad joke. I don't care. Long stack, short stack, medium stack, TCP/IP stack. It was worse than my usual. Worse than your usual? There is no worse than usual. TCP/IP stack, y'all. Now then. Okay, that's better, isn't it? Okay, so TCP/IP stack. So we talked before about development of TCP/IP and that was in the early, late 70s, sorry, and the implementation of ARPANET ran out in 1983 when they switched to TCP/IP and it is the standard that we continue to use today. However, there's a model that you may have come across called the OSI model, and the OSI model is a very handy way of thinking about different layers of the Internet. And there's lots of references to the OSI model, but technically, TCP/IP does not have as many layers. So the OSI model has seven layers. The TCP/IP layer combines some of those to have four layers, but it's intentionally less rigid, which is why TCP/IP, I think generally it's accepted because it's less rigid is far more popular, to describe in these terms. So anyway, not going to talk too much about the OSI model or references. I'll only reference it where it's relevant sticking with TCP/IP. So the four layers in a TCP/IP layer stack are the application layer, the transport layer, the network layer and the data link layer. So what the hell does that mean? A layer in the stack refers to the set of protocols that are used whether they are hardware or firmware or software whatever you want to think however you want to think about it those different layers control how data is handled and once I handled it could be the lowest level of of what voltage this is indicating what a one is and what a zero is, all the way up to, this application is requesting ASCII character, whatever from this site, and it's encapsulated in a packet and that packet switched through, blah, blah, blah. So every layer of the TCP/IP stack has a specific purpose. So we're gonna talk about each of those layers in turn. So we'll start at the top and work our way down. So the very top layer and the application layer in TCP/IP is actually the amalgamation of three separate layers in the OSI model. So the OSI model starts with the application layer. At the top, it then calls a presentation layer and a session layer. But in TCP/IP, they just bundle that together and call that the application layer. And the reason they do, I think, is because session and presentation are all essentially subsets of an application. And when I say an application layer, I'm talking about now the protocols that operate this layer, the ones that we're all familiar, I think most of us are familiar with like HTTP, Hypertext Transfer Protocol, FTP File Transfer Protocol, SMTP, which is the Simple Mail Transfer Protocol, and SNMP, Simple Network Management Protocol. And I did warn you that there are a lot of acronyms and that, there you go. So transport layer, next one down. Essentially, the transport layer has two primary protocols. And this is where we mentioned our good friend TCP, the transmission control protocol. But there's another one UDP, which is the user datagram protocol. It's all about protocols, y'all. So TCP, what it does is it'll take the data from the application layer, and it'll break it down into the quote unquote, correct sized pieces. But it'll handle the release of those pieces onto the network, it handles the acknowledgement of received packets and manages resends of lost packets if there's timeouts for said sent packets. Now, UDP, on the other hand, is a bit more streamlined. And by streamlined, I mean careless. And by careless, I mean, it just doesn't care quite as much. And that's the way to think about it, because it doesn't really take any measures to ensure the data sent is actually received by the target host. It's like, send it into the void, baby. See you later. Now, Predominantly, it's useful for lower bandwidth, lower priority tasks like streaming audio and video, you know, including Skype and VoIP, you know, where if you lose a small number of packets of data, you're still going to be able to figure out what's going on. Mind you, you know, the discerning person would say, I need every last bit, or it's just not right or something. I don't know what people like that might say, but that might, they may say that. Anyway, so yeah, UDP is generally frowned upon for anything other than streaming. There's probably other applications for it, but that's the most common one that I'm aware of. - Yeah. - Okay, network layer, also referred to as the internet layer. Anyway, and that's how packets are actually routed around a network, figuring out how to reach the correct destination. Then of course, there's the data link layer. And it's also referred to as the network interface layer, But in the OSI model, it's actually two separate layers. The data link layer, again, same name, but also the physical layer. And the physical layer is literally the voltage level. So voltage this, current that, blah, blah, blah. Now, the reason I think that the TCP/IP model is better is because it, again, it can combine those levels of the OSI model that make the most sense because they normally get grouped together. Because ordinarily, the physical layer and the data link layer are combined anyway. Okay, so the basic structure of the data, as well as the voltages kind of go hand in hand in terms of their implementation. So I think that that makes sense. So those are the basic four groups, and a little bit about each of them. So now, it's time to talk about the data link layer of Ethernet, which is essentially the building block of all networking that the internet is built on. But the data link layer, the data link layer level, the level, the linking level layer of the data link. Lots of L's. Okay. So in the OSI context, the physical layer of Ethernet didn't start out as those pretty blue. Is it pretty? well, they're blue anyway, Ethernet cables that we think of today, those Cat5 patch leads, whatever you want to call them. It actually started out as black, typically, coaxial cable. Have you ever seen a coaxial cable Ethernet network, Vic? I have. And how did it strike you? Well, aside from the connector on the end, it looks like any other coax cable. But was it cool? Did you walk up to it and say, man, that's some pretty cool coax you got there? I do recall thinking, well, that looks much easier to connect than the way we have to screw that thing onto the back of the TV. Yeah. Yeah. I'll admit the first coaxial Ethernet network I saw at uni, what struck me about it was how shiny it was. Because the parts that you could see the majority of where the coax would come out of the cable duct, out to the back of the computer and go into a shiny T-piece. and then out the shiny T-piece sometimes into a bus terminator. All of it was a shiny silver either BNC connectors on a T-piece or BNC connector into the back of the computer network interface card. It was all shiny. So I actually saw that and thought, "Oh, that's cool." - And then of course back in the early days-- - It's just sparkly. Yeah, very sparkly. And back in the days when I was still learning about how the stuff worked, every now and then I'd be like, "Oh, this N-bit, the terminator, this N-bit comes off. Oh, cool." Take it off. "Oh, that looks cool." And then down the other end of the rail computers, people would say, "Hey, something just happened to my data transfer." I'm like, "Quickly and surreptitiously put that back on again." Whoops. Sorry. They tend to frown on that. Whoopsie doodle. Anyhow, bottom line. Thank you, Ned Flanders, for that. Anyhow, so yes, I've kind of given away some of that. So, given away. Given away? No, I already talked about it before I was going to talk about it, but it's down on my list to talk about, which is BNC Connectors. So BNC is just a kind of connector. It's very typically used on coaxial cable. The funny thing is that the coaxial cable they used, it was, they typically used 50 ohm coax. And the best part about 50 ohm coax is that's just the same kind of coax that you use in CB radio and on amateur radio for radio applications. 75 ohm is actually far more common back in the days of television coaxial cable. I guess some TV still use coaxial cable. So 75 ohm was the standard for TVs for a whole bunch of different reasons. And I'm not talking about the open ladder line, which is 300 ohm and that's balanced and this is unbalanced and nevermind that. So the attractions of coaxial cable, why did they start with coax? Any ideas, suggestions, thoughts, comments? - I don't know. - All right, fair enough. Maybe it's obvious, maybe it's not. I guess I've maybe been doing this too long and I think everything's obvious and it probably isn't, but you know, the bottom line is that coaxial cable, if you do a cross section of coax, what you've got is you've got a core, a center conductor, you've got what they call a dielectric and the dielectric is just an insulator. So fancy name for an insulator. Got my dielectric in the middle, y'all. And I don't know why I keep putting on that accent. It just feels cool. I just, I don't know. Maybe I need to upgrade my accent. Anyway. I think I've had too much coffee. This is my problem. - Maybe that's it. - Sorry. It's all good. It's all good? Okay, cool. Very good. Okay, so, right, yes, and then outside that you have a shield. And the shield can, is most often a flexible braid, depending on the grade of the coaxial cable, sometimes you'll have a foil over the top of that, and then you'll have an outer protective sheath, and that will be another insulator. So the idea is that your center conductor is heavily shielded and heavily protected, which makes it very quiet. Not just that, the dielectric is essentially such a good insulator and such a good, it's such a uniform spacing between the sheath and shield and the center conductor that it has a very highly controlled characteristic impedance. And the one thing that's really important in transmission lines for efficiency is having a consistent characteristic impedance of 50 ohms. Because what that means is that you can model the characteristics of that line. And that means that in theory, at least, you can get more data. I say in theory. Okay. However, the ultimate solution is that we didn't use coaxial cables, we've gone to twisted pair. Why? We'll get to that in just a second. The first ever coaxial Ethernet was actually called 10BASE-5 and it used RGA8X coax and it's rather thick, not particularly flexible, probably good for whipping people with but that's a pretty sinister idea and it's expensive. So for those reasons, actually was referred to as thick Ethernet. These days you'd be thick if you used it. Following that in the mid 80s came thin Ethernet. Some people called it thinnet, but technically 10BASE-2 and that used much cheaper RG58 cable and I used a bucket load of that. Not that I ever caught up and put in a bucket, but anyway, I used a fair amount of it. And that my early radio days and CB radio days, RG58. That's for the win. So still, however, dealing with 10 megabit per second transfer rates. And that's where 10BASE-5 started, hence the 10 at the beginning, right? That was the 10 megabit transfer rate. So 10 megabit, that's so fast, man, blistering speed. Okay. Now at this point in time, all the coaxial cable was connected together at the back of all the network interface controllers. So what's the one obvious observation from connecting all the computers to a coaxial cable, and joining them all together in one long bus? The conclusion is? A traffic jam. Pretty much. Yeah. You got a situation that really isn't all that much better than what we have now with 3G and 4G. You've heard me lament in the past the problem with everyone saying, "Oh, we'll just, you know, add more data to our Wi-Fi, add more data to our Wi-Fi." It's like, well, eventually you're going to run out of bandwidth, guys, you're gonna run out of radio spectrum, it's going to be a problem, okay? Whether or not people like to believe that or not, it is absolutely going to happen. However, if you can contain all of it inside a cage of sorts, like a coaxial cable, that's great. But the problem is If you put every computer that wants to talk on the same bit of coaxial cable on the same bus, you got yourself a problem. So ultimately, it was not technically a point to point link, it was an Ethernet bus. Okay. Okay, however, in 1984, a little company called American Telephone and Telegraph, that'd be AT&T, they created something called Starland, which was way ahead of its time. I say way ahead, I mean a few years, but still ahead of its time. And it used far cheaper twisted pair cable in order to carry the data around in, well, you guessed it as the name suggests a star topology as opposed to a bus topology. And as amazing as it was, it never took off. In fact, it was complete failure and collapse and no one ever really used it. Shame that. But the concept and the ideas behind it essentially drove pretty much all of the future wide local Ethernet networks. And that's what we use essentially today. The bit rates originally, were pretty ordinary. And maybe this is one of the reasons why it failed one megabit per second. I mean, really, that's like one 10th the speed of that thicknet, or even thinnet. You know, give me some thinnet. I don't want this Starland rubbish. Anyway, you can keep the one megabit per second, shove it. Okay, so it wasn't actually until 10BASE-T and that's the one that everyone remembers is 10BASE-T. That came in, in the early 90s. So when I went to uni in 1993, I think it was, my memory's hazy. Anyway, the PC labs are still connected by coaxial, but within one year, they'd all switched over to 10BASE-T. Why? It was easier. It was simpler. It was cheaper. easy to install, it was just better. It only had the same speed, but of course, because of the architecture, it had a better effective throughput. It wasn't a bus anymore. So, well, it was and it wasn't. It had the capacity to not be a bus, and we'll talk about that later. So 10BASE-T originally used CAT3, which people don't remember. But anyway, the far more popular standard, however, was called category 5 and that category 5 cable essentially is a 4 pair unshielded cable and it uses an 8P8C modular connector. However, in telephony as to say telephone, telecommunications, the 8P8C is referred to by another designation that most people are more familiar with and that is RJ45. So the RJ45 connector and the Cat5 cable has become ubiquitous with wired of Ethernet connectivity and has done since the early 90s. So the early standards 10BASE-T define different modes of operation, full duplex, half duplex, and auto negotiation for originally, of course, 10BASE-T, then 100BASE-T, and in more recent systems, gigabit, 1000BASE-T. I'm not gonna talk too much about all the auto negotiation, full duplex, half duplex, all that, 'cause I don't think it's all that interesting, but there's plenty of links in the show notes if you'd like to. And on a sort of a sub note about the amount of notes in the show notes for this episode, there is a metric ton, metric? Even an imperial ton, doesn't matter. There's a lot of stuff on the internet about the internet. So there is no shortage of information out there about this. So if you really wanna know about full duplex, hard duplex, auto negotiation at that low level, go for it, plenty to read. Anyway, one of the other interesting things is that cross that 10/100 base T only use two pairs actually in a standard Cat5 cable. And the pins that they present on are pins one, two, three, and six. The other two aren't actually used in standard installations and for people that were being really stingy many, many years ago, there were actually patch cables that came out that only had two pairs in them because it was cheaper, right? And the super ultra cheap ones, they'd say, "Oh, I'll just plug this into my awesome IP phone." And you plug in your IP phone and we'd be dead. And you'd be like, knock, knock on the phone. "Hello, why are you not working?" And it's like, "Oh, it needs power over ethernet. And the power over ethernet uses the other pairs. Oh, so I've got to go and upgrade to a real ethernet cable." So yeah, beginner's mistake during that transition period. But eventually, of course, like anything mass produced, made so many of these damn patch leads you can now pick them up anywhere you like. Monoprice probably got them. MSY here in Australia, dirt cheap, you know, like three bucks, two bucks, and you get a five meter cat five patch lead. It's like, it's insane. These things are mass produced by the metric ton. There I go again, imperial ton. It really doesn't matter. A ton is still a lot. So, high tonnage. Okay, Gigabit, however, uses all four pairs. And it also requires a higher quality of cable with tighter tolerances. And they specify that you, well, they suggest that you use category 5E, which is a far tighter tolerance. Now, despite this, irrespective of the kind of cable that you use, the amount of inductance and capacitance you get with a twisted pair, It's the same problem with phone lines. It's just on a smaller scale and you're pushing more data. Think about it. I mean, ADSL, maximum data rate in ADSL 2+ is what? 40, 60 megabits per second. Well, we're trying to push 100 and a gigabit per second. Obviously, you're going to have a range limitation. Obviously, the capacitance is going to kill it. So what happens? 100 meters is typically all you get. But that's fine because that's all you've got. I mean, imagine how many houses have you got that require more than 100 metres for a point to point connection? Not many that I can think of, unless you own a mansion or a McMansion or several McMansions connected by a McMansion corridor. I don't know. Why? I mean, I don't know. Do you have a McMansion? Do you know what a McMansion is? I do not have a McMansion. I do not know what a McMansion is. I'm assuming maybe possibly a mansion owned by somebody that owns McDonald's. No, maybe that's an Australian expression. I didn't think it was, but the idea of a mansion is that a mansion is like really, really big and expensive and luxurious. And, you know, usually has two floors and it's a huge amount of rooms. Most of them unnecessary. Most of them unused requires dedicated full-time cleaning because it's so ridiculously oversized. It's a sign of opulence and wealth and all that other, you know, stuff that, you know, you could argue in a capitalist economy is a good thing, but most people look at that and shake their head and say, whatever. Anyhow, A normal house is like what you and I live in. It's a house with a normal number of rooms that are all got stuff in them. Some would say too much stuff, especially in my house. A lot of stuff to me, stuff, toys and kids rooms. Anyway, a McMansion is somewhere in between. You got enough money to like, and they usually are like a McMansion. So like it's more of a cookie cutter feel because mansions are typically very highly customized. whereas a McMansion is still built by a reputable company, but they'll build lots of them and there's only a limited number of floor plans. And yeah, so hence McDonald's, McMansion, I don't know. So at least that's my understanding of the entomology. I've done no research on that one. So there you go. Could be wrong. Ask Wikipedia. Okay. - So it's also not related to a Scrooge McDuck money bin. - No, it's not. - Okay. - Okay. - I'm glad we cleared that up. - Yes, we have indeed. Clarification has been sought. Okay, good, lovely, charming. However, if you use high quality cables, it's possible that you can push it to 150 meters, but higher speeds aren't guaranteed. You cross the 100 meter limit, you're on your own. Now, there were some cases that I've done installations where what we did is we actually put a ethernet switch at roughly the 100, 110 meter point. So I did some factory automation about 10 years ago, where we had a 10/100 link that we wanted to get from one end of the factory to the other. Total length of the factory was around about 180, 190 meters. So the computer that they had in the office at the front of the warehouse was actually communicating with a PLC down the back end of the warehouse connected to the machine. So what I wanted to do is I wanted to have a look at what the machine was doing, but I wanted to do that from the beautiful air conditioned comfort of the shop front. And they could say to their customers coming in, "Oh, look, here's our machine making widgets." And customer would come and look at the screen and say, "Oh, look, it's pretty widgets, lovely." Anyway, so in order to make this work, you know, we put an ethernet switch in between. And 'cause, you know, the good news was that there was a GPO PowerPoint power socket, whatever you wanna call it, that was convenient. And along the way, we just put in a junction box for protection and hey, presto, Bob's your uncle and hopefully not your auntie or whatever. So I said, why twisted pair? Well, it's cheaper than coaxial cable, but it doesn't have the control, the tight tolerances that you can get with coaxial, but the cost tends to win. Plus there's something you can do with twisted pair that you can't do with coaxial cable. And that's something that's referred to as common mode noise rejection. Common mode noise, the idea is that if I run two signals and one is the mirror image of the other signal, one on one wire, one on the other wire, and then what I do is I am exposed to noise. That is to say the cable's tightly twisted together and I get noise on that cable. The noise is going to affect each of the conductors equally or essentially equally. Therefore, it should be possible because by comparing the two individual wires, signals, I should therefore be able to cancel that noise out through a mode using an operational amplifier. You can actually cancel out the common mode noise. You'd be surprised how effective that actually is. - That's cool. - It is very cool. So common mode noise rejection is very, very handy. And it's the sort of thing that has made twisted pair very popular in even in industrial applications. So many Modbus networks, you will just run on standard twisted pair cable. A lot of analog instrumentation cable is run on Twisted Pair. Straight single core cable in industrial applications for analog signals is essentially not done. Everything is done as a Twisted Pair for that reason. And sometimes we've had overall screens, individual screens, and this is not an industrial automation episode, that's another day, nevermind. Maybe it isn't another day, who knows? Okay, so Twisted Pair, not so bad as we thought. Range is limited, certainly, but then so is coax. And frankly, it's a heck of a lot cheaper. and that gave it the edge in the end. So, oh, and it also, yeah, I haven't talked about point-to-point yet. Yeah, okay, so we'll get to that. Okay. Next, next, next, next, next, next, okay. I'm gonna talk a bit about Ethernet switches and point-to-point links, but I think it's time for us to step up a level and get a little bit higher than the data link layer and start talking about addressing. - Okay. - Okay, Mac addresses versus IP addresses. So, so you got a bunch of devices, right? But you got to know how you get a message from one to the other. It's not just enough to connect them together, whether they're on a bus, whether they go to a switch, you know, either way, it doesn't matter. You still need to know how to get the data to where it needs to be. There's no question you need to have an address of some kind. No different to houses in the street. You can't just say, go to Bob's Road. Well, what number on Bob's Road? Well, it could be from number one up to one million. Mind you, that's a very long road, but still, you got to know. You know, Bob's House. Yeah, Bob's House, man. Totally. You know which one that is. So anyhow. OK, so what the heck is a MAC address? You maybe have heard of MAC addresses, maybe you haven't. I'm sure most people have. And MAC address has nothing to do with McDonald's. So fine, let's get that out the way. It stands for Media Access Control. So named actually because the OSI layer, because the third, well, not a layer, but a media access layer is a group of the bottom three layers, I think of memory serves as the bottom three. Anyhow, media access control is a physical segment address and that a uniquely defines that connected device. So practically every piece of computer hardware prior to a decade ago used burned in, otherwise referred to as hard coded Mac addresses. However, some hardware now has the capability to refresh and modify its Mac address. Usually through a firmware update, sometimes it's stored in flash memory and so on. Some of the hardware that I deal with has them configurable through DIP switches and jumpers. Not the whole Mac address, mind you, just a portion of it. And some do the same for IP addresses. Thankfully, it's not many. Anyhow, automation sometimes is annoying. So MAC addresses come in three types. MAC 48, EUI 48 and EUI 64. EUI actually stands for the extended unique identifier. And that's IEEE lingo. That's the Institute of electronic and electrical engineers, whatever, them, I think. Oh, I think that's what it stands for. Oh gosh, something I didn't look up. There you go. Anyway, so the first two, Mac 48 and EY 48, they're used in ethernet, wifi, Bluetooth, for example. So the first two types, they're used in ethernet, wifi and Bluetooth, just for example, whereas the 64 version is used for IPv6, which is utilized in Firewire and Zigbee, you know, for example. More on that later. Okay, so it's all about octets. Do you know what an octet is? - Is that a set of eight? - Yes, of course, oct, oct, eight, octopus, eight. Yeah, it's kind of a giveaway, but what annoys me is that octet is actually just another way of saying eight bits. Okay? - Okay. - For those that slept through computer class, that's what it means. It's like, it's another word for a byte. But you say an octet because byte might be confusing. I don't know, whatever. So, okay. How many octets do you see? Six by eight is 48, eight by eight is 64. This is not rocket science, it's in the names, people. So you got Mac 48, EY 48, all that's got six octets. and EY64 has eight octets. Again, like I said, not rocket science, it's in the name, pretty damn straightforward. So the first three octets, they're assigned to a manufacturer by a governing body. In this case, it's the IEEE's Registration Authority and they administer the assignment of the identifiers and these are referred to as the OUI, the Organizationally Unique Identifier. More acronyms, there weren't enough. So there's a link in the show notes to the IEEE Registration Authority Committee where you can apply for your own. Best of luck, enjoy yourself. In the case of Mac EY48s though, the last three octets and the case of EY64, the last five octets, they're defined by the manufacturer. Some manufacturers have a method, some don't. Bottom line is they should all be unique. So let's do some numbers, shall we? In theory, and this is in theory, this is not just maximum number of permutations. there are for EUI 48 address combinations, there are 281 trillion potential combinations. You think that's impressive? How about this? For EUI 64 address combinations in theory, there are 18 and a half million trillion possible combinations. That's a lot. However, those numbers are in fact BS. And the reason that BS is because the reality is that a a single manufacturer is only operating under a single or a subset of a restricted number of the first three octets for either system. So ultimately, therefore, if you think about any one manufacturer who's assigned a single OUI, then you only have 16 and a half million possible combinations for any EUI 48 address, or for 64 address, you have one trillion or thereabouts combinations for a single one. - Is that all? - Yeah, that's all man, have 1 trillion. Okay, so when Apple ships it's 1 trillion Mac, it's screwed. Of course it's not, they'll just get more RUIs. Anyhow, if they probably have more than one RUI already. So, you know, and I guess that's the point, isn't it? Larger companies, you know, that manufacture the network interface cards and the chips and companies like Apple and Dell and whoever else, they're gonna have multiple RUIs. So, you know, it's not gonna be a problem, but you know, back in the early 2000s, myself personally, I actually had a MAC address conflict with some control system hardware. But the reason was that it was actually factory firmware flashable, and they had mistakenly flashed it with, you know, identical MAC addresses for reasons that I have absolutely no idea what they were smoking, drinking or sniffing at the time. I don't know. But anyway, and it took me a few hours to find it because it's not the sort of thing you think to look for. It's like, No, MAC address is unique. I'm not going to, this is not going to be a problem. You should take it for granted. Yeah, exactly. My IP address is different. No problem. It's a bit, why am I with the, the, the, I'm getting lost packet. What the hell's going on? Why is it? This, I don't get it. Stupid. I'm going crazy. Anyway. Uh, it was easily fixed. Once I easily fixed it, it wasn't easy, easy, but at least we did fix it, which was, you know, just applying a firmware update and away you go. Okay. IP addresses, not Mac addresses, IP addresses, they come in two flavors, IPv4 and IPv6. And they're represented decimally as opposed to hexadecimally. So you'll see IP addresses referred to by numerically, zero to 255. Whereas Mac addresses, I didn't mention this before, but they always seem to be represented in hexadecimal, which is zero, zero to FF for each of the octets. Loving the octets. Anyway, so V4, four octets, V6, six octets. Anyway, honestly, I've covered this previously a little bit in episode 16, where friend of the show, David Legate, wrote a very nice piece looking into the IP numbering and quantities. I refer you to both of those, and we're going to move on. Before we do go into Ethernet switches though, just a quick note that there are special IP addresses have been put aside for things like broadcast messages. And usually x.x.1, hang on, x.x.x.1 is reserved for gateways routers usually. And everyone's favorite, the loopback IP address is, drumroll please Vic. Okay, actually I'm going to put that the other way around. I'll give you the drumroll, but I can't make that noise. How about you just tell me what the loopback IP addresses? I have no idea. Seriously, dude. You tell me localhost and I'm going to slap the mic. Okay, it's 127.0.0.1. That's your loopback IP address. I thought you knew that you're a developer. I would refer to that as localhost. I haven't done a lot of networking stuff. Okay, fine. You just didn't. I know IOSA, I suppose, yeah, okay, fine. Okay, fine. - I'm a code monkey. - You're a code monkey. (both laughing) - I haven't done a whole lot of networking. - Okay, fine, fine. - I just plug it all in and get it all working and then forget about it. - Who is it someone said to me once, just set your IP address to 127.0.0.1 and you'll be fine, then you'll connect to the network fine. Just, I'm like, you think I'm gonna-- - It doesn't sound good. You think I'm an idiot? Don't answer that. Anyhow, that's what I said at the time. It's still not funny, but anyway, there you go. So yes. So yeah, loop back IP address. Stop talking to yourself, stop talking to yourself, stop talking to yourself. Moving on. In fact, actually, you know what? Before we do move on, very important. We need to talk about our second sponsor, and that's SapientPair. Now, SapientPair have decided after years and years of being annoyed with existing to-do apps when they're shopping, like grocery shopping or shopping for anything actually, they would start to create an iOS app for the iPhone and it's called Shopee. There's a ton of to-do list apps out there and I've used a lot of them over the years, but going shopping is a very specific use case for a list. And if you're shopping for more than just yourself, then Shopee really begins to shine. I mean, the best way to describe Shopee is it's a fully collaborative shopping list app and it's simple and easy to use. I picked it up and figured out how to use it immediately. It's not cluttered with options. It doesn't presume that you live in a specific country or present you with a hundred options for butter or milk. You just type in what you want to remember to buy in the list. Enter an amount if you want to, that's optional, and it's there in your list. And it remembers what you've entered for future reference, even in the order that you buy them as you walk through the supermarket. So that's cool enough. But then when you share your list by email, iMessage, so on to your spouse, your partner, your kids, and hopefully that'll just add chocolate and ice cream to the list, You can add, mark off, reorder items in the list as they need to. Anyway, I've tried this in real time. Between two iPhones, the sync happened in less than three seconds and that was over 3G. But I also love the pocket lock feature. Now, if you're security conscious like I am and you've got a passcode set, there's nothing more annoying than having to lock your phone, slip it in your pocket, and then get it back out again at the end of the aisle that you're walking down in the shopping center just to unlock it again to look at the list. Well, Pocket Lock disables the screen when it detects it's in your pocket and it re-enables the screen as soon as it's removed. No passcodes necessary, no fuss, it works really well. Now, my wife and I have used it several times and where we used to note things in reminders or to-do apps or even on paper. Anyway, now when either of us go shopping, we use Shopee. Open Shopee to indicate that you're about to start shopping and then the the geolocation detects the store that you're shopping at. And on our shared list, the other person will get a notification that you're about to start shopping. And then if they remember that they need you to grab something for them, they can tap the notification, go straight to that shared list, and they can quickly add that item. It'll then appear on my list while I'm in the shop, and I can grab it while I'm there. It's brilliant. Those last minute, "Is there anything else you need?" phone calls, you just don't need that anymore. Okay, so Shopee is free to try for the first month with no ads. After that, it becomes ad supported. There's no risk, no loss of functionality. But if you wanna help out the developers, you can in-app purchase three or 12 month ad removal for $1.99 or $4.99 US respectively. The update to fully support iOS 8 and the iPhone 6 and 6 Plus has been live on the store now for a little while, and it features a handy reachability feature where you pull down to add new items. There's been a fresh coat of paint, and now you can move checked items to the bottom of the list to declutter the longer lists if you want to. So please visit this URL to learn more. It's sapient, that's S-A-P-I-E-N-T dash pair, as in two, .com/pragmatic, and follow the links to the app store from there to help out the show. You can search for the app in the store, but if you use that URL in your browser of choice, it will help out the show. Thank you once again to Sabian Pairs Shopiac for not only making my shopping life a little bit easier, but also for continuing to sponsor Pragmatic. Okay, ready for some switching? I am. I'd say let's switch it up. That's a good one. That's good, Arthur. That was terrible. Okay, Ethernet switches. So, I'm actually really interested in this part. You have Ethernet? Okay, Okay, so Ethernet switches essentially come in two flavors, a layer two, which is a data link layer switch or a layer three, which is a network layer switch. We'll talk a little bit more about layer two switches specifically later, but before we do, let's talk about what a hub is and isn't. A hub is not a switch. Now, the funny thing is that when I first built an ethernet network, we couldn't afford a switch. We can only afford a hub because hubs were so much cheaper than a switch. So what a hub does is when a PC, or I should say PC, when I say PC, I mean a Mac or a PC, I'm not being, you know, brand biased or anything. I'm just, you know, it's a personal computer, okay? So get over the connotation that the connection in your brain and PC equals Windows and doesn't. So PC on one port sends a packet, the hub then repeats that packet to every other port that's connected to it. Okay, so it's pretty dumb. It's just a hub. - So it does. - Connects everything together. So essentially it creates the Star LAN idea is essentially it creates a point to point link and it aggregates all the point to point links for all the devices connected to it and turns it into a shared bus in a manner of speaking. So it acts like a repeater. Which, you know, is good, is bad, is whatever, I don't know, bad, I guess, because the problem is that if you've got a relatively large network with a lot of hubs, it'll saturate very quickly with traffic. And the bigger it gets, the worse that gets because there's a lot of repeating going on. Yeah. Okay, so there are very subtly different effects when you compare an Ethernet 10BASE-T network using hubs versus a, let's say a 10BASE-5 network using a coaxial system on a common bus. And those are very subtly different effects, but really it's the same kind of problem. Okay. Now I mentioned routers before, Just quickly touch on that. Routers are designed essentially to be gateways that bridge two networks together. And typically they've been used for ISP connectivity, connecting to a business network remotely via, let's say a virtual private network, for example, into a business, say you're at home, you're not connecting to your business, the company you work for, whatever, then you would do that through a router essentially. But hubs and switches, switches are a very specific beast. And as I said, they were very expensive. And these in the beginning, switches are very expensive 'cause they required more processing and there's a good reason why. I actually seen a hub come to think of it for quite a while for sale. It used to be there for a while though, as I said, so much cheaper. And there'd be a handful of switches available in your IT store of choice. And now I think hubs are pretty much gone. When's the last time you've seen a hub? - Like in the store for sale? I don't know that I've ever seen one. - Really? Wow, okay. - Yeah. - Well, I know you're older than me, so. - Yeah. I've seen some in use at some places where I've worked before. - Okay, right. Fair enough. Okay, so nowadays everything is a switch and switches are far more efficient. Why? That's the question. Okay. Simple answer. An Ethernet switch learns and figures out which port to send the packet onto. So unlike a hub, it just repeats it everywhere. A switch will selectively send the data, the traffic over the port that's gonna be where the destination that's connected to the next link or to the destination directly, either or. So Ethernet switches are good because each link is handled independently. So each port on an Ethernet switch is dedicated to an individual computer, which means that you can control it independently. So let's say you've got a 10/100 Ethernet switch, but the device connected on one port, it's stuck at a hundred. The other ones, the next device connected to the next ports is stuck at 10/100, 110, whatever in alternation. Yeah, it can negotiate each of those independently without any interference to each other. It can also handle, some of the good switches can handle auto crossover as well. We talk about crossover cables, why would you need crossover cables? Well, because one pair is for transmit, one pair is for receive. So what you'll do is you'll connect it into one device, you connect it to another device. They're both expected to be, and devices connected to a switch or a hub. Therefore, you expect the transmit and the receives to be on the same pins, like computer to computer. - You mean it's not because PC makers hated me? - No. So you would plug in a standard straight through cable between two devices and they wouldn't be able to talk to each other on Ethernet. In fact, that's still probably the case. But they expect to be plugged into an Ethernet switch. So the Ethernet switch would be the other way around. So the problem was of course is that everyone would then buy a - the computer nerds that wanted to go point to point between two computers without a switch and form a private network usually for gaming like playing Net Doom, Net Quake, Net whatever. You know, like I used to do sometimes, occasionally maybe. automation as well when you've only got one computer talking to another computer for the purposes of data transfer and so on and so forth using lap link software for example. Yeah. All of that sort of good stuff you'd need a crossover cable. I think the ports on the Mac are smart enough to figure it out now aren't they? Yes I think so and most modern switches are these days they can handle auto crossovers they'll automatically detect which has got transmitting and which is which is receiving which is cool and handy so you can use any old cable you like. So anyway, switches are good like that. Now, speed negotiation. So let's say you've got a gigabit switch, assuming you know we've got, you know, okay, so everything comes down to you've got to have all the pieces in the connection for that to work. So to have a gigabit connection, obviously you need a cable that's got all four pairs in it, first of all. Second of all, it helps if it's Cat5e. Not absolutely necessarily a requirement, especially if it's over a very short distance, but you really probably ought to. Because otherwise, if the length is too long between them and you plug it in it's not a Cat5e cable, you're not guaranteed there won't be a tight enough control of the impedance on the transmission line for you to actually achieve a low enough noise to achieve gigabit. So anyway, bottom line, use a good quality or relatively short low quality cable and you'd be right for gigabit probably over short distances, but still go cat 5e conquer wrong then. And then not that much more these days. I think most cables most patches now are essentially cat 5e is relatively standard unless you get the cheap, cheap, cheap, cheap, ultra cheap ones. Okay, so you got gigabit on each end, plus you got a gigabit rated patch lead, then both ends can negotiate and say, Yep, my switch says I'm gigabit. My device says yes, I'm gigabit. And they happily shake hands and say, let's go gigabit, baby. And you're often right. However, sometimes the wheels fall off. There's too much noise, you've got a bad connection, or unfortunately the device you're connecting to is only a 100 megabit device, in which case you're stuck with the lowest value. Well, the next, hang on, the highest value that's capable of each end or the interconnecting medium. That brings you then stuck to 100 meg, or maybe it's real bad and you're stuck at 10, or worse than that, you're stuck with nothing. So the speed negotiations were the very first things that happens. Once you've got that basic speed negotiation established and your auto crossover has been dealt with, that's when the Mac address is identified. So you say, the device says, this is my Mac address and the server then makes a note and says, right, this port, port 10 on a me, me being the switch me, port 10 is connected to this device, this Mac address. And it adds that to its internal table. - So-- - That would be self.port10? - The naming mechanism will vary by manufacturer, but essentially it maintains its own table with the port table that shows a list of what devices are connected to what ports. So some switches also employ something called MAC address locking. And I've talked about this previously on episode 27. There's a link to that in the show notes. And MAC address locking is that effect whereby you can only plug that specific device with that specific MAC address to that port. Otherwise alarm bells go off, the communication will get locked down, no data will go back and forth, it'll get to the auto negotiated, it'll agree a speed and you'll say, Oh, hang on, not you're not my friend, go away. And in my case, that's when all the security personnel came down to my desk and said, What the hell do you think you're doing? I'm like, sorry, anyhow. And if you want to know more about that, listen to episode 27. I'm not going into that again. So once all of that has happened, all of the levels of negotiation, you're a crossover, the MAC address has been handed over, if there's MAC address locking, the MAC address lines up, it's at that point, when IP addresses start to go back and forth. Okay, now, when you want to send a packet of data, and we'll talk about IP address allocations in a minute. When you want to send a packet of data to someone on your local network, the ARP, which is the address resolution protocol is used. And it's actually based on MAC addresses first, not IP addresses. Because these are the switches. So yeah, the switches we're talking about, these are layer two switches, right? Network level would be layer three, that would deal with IP routing, not layer two, layer two is MAC, it's the actual closest to the physical layer. Well, it's one step above, right? Data link layer. So now there's a great article I've linked in the show notes on how IRP works in nauseating detail if you really want to know. And I encourage you to check it out if you are that interested. I however, am not 10,000 foot view, maybe 1000 foot or even 100 foot view, but certainly not that level of detail. So packet has destination IP address device be sent from device a to the ethernet switch. Maybe I should say that again, a packet with a destination IP address that is the IP address of this device B is sent from device A and it goes to the Ethernet switch the device A is connected to. The Ethernet switch then checks to see if the destination IP address is in its table. It notes that it is and directs the packet to that device's MAC address specifically via that port. If it's not in its table then it sends a broadcast message to all ports asking to know if anyone knows what the MAC address is of the IP associated with that IP address. If there are no responses, it tries gateways and routers and so on. Hopefully someone comes back and says, "Hey, it's me." And if they do, it's forwarded on accordingly. However, if no one comes back, then the packet fails to send. Now, the simplest example of how you can see this in action is a ping request. So you can go to the terminal or the DOS prompt and type in ping, your IP address of choice, and see what happens. If you get a response, then the switches and routers found a path and they'll give you a response. If not, you get "fail, fail, fail, the world is ending. Oh my God, it's the end of everything." Or something like that. I think it's actually far less dramatic, but still. Okay. I don't know if I can't connect to the internet, I consider it the end of everything. Yeah, I think a lot of people do. And that's totally and absolutely fair to the first world problem. So it sounds like I'm jumping around a little bit, but I'm trying not to, but a lot of this stuff builds on prior knowledge. So the problem is knowing where to break in and say, well, we're going to start here. So now that's the basics of the basics of ethernet switches, the basics of how that information is routed within a local network. I sort of stopped at the router a little bit. So let's talk now about behind a router and let's talk about essentially, well, let's talk about subnets. Now we're going into the 200 level classes. I don't know, maybe, maybe not. Let's see how we go, shall we? Subnets are critical to understanding how it's possible for networking to function on an enormous scale like the internet. So, I mean, imagine a world just for a second where every single Ethernet switch spoke to every other Ethernet switch. Every time a packet went out, just imagine it just it wouldn't work. It'd collapse. Right. How we reduce that traffic load between every other Ethernet switch in existence is we create subnets, little miniature networks that attach to the side of the Internet, if you like. You know, this is like a zip code versus your actual street address, right? Yeah, kind of like that. And the access portal to the internet has a single IP address, but the router that connects you to that single IP address then acts as the gateway and it forwards on those packets, but it remembers who those packets came from so they can pass them back when the packets return. But most subnets will fix the first three octets in IPv4 and that gives you a total of 254 possible devices. I say most, I mean there's actually a little bit more to it so we'll get to that in a sec. And that's fine for most households but businesses usually will require more and they'll fix either the first one or two octets. And that's why a lot of home networks are going to use 192.168.X.X and usually it'll be just 0.0.X or 0.1.X and that'll be that. How did that come to be is the question on everybody's lips. Well, maybe it is, maybe it isn't. It was a question on my lips, but then it's because I'm a bit weird. That's okay. The IETF, the Internet Engineering Task Force, sounds a bit like Justice League or something. I don't know. Maybe it doesn't. I actually don't read too many. They're my heroes. Yeah, I don't read too many comics, but for some reason that just IETF sound like some kind of a task force, like it's just some Justice League thing, whatever. I don't know. Anyway, they published RFC 1918, which was not published in 1918. It was actually published in February 1996 requesting allocations of specific IP address ranges for private internet. And the reason that they reserved these was so that these IP addresses would never be allocated in the public space. So they're only for private internet or intranet actually is a more accurate description. So this would mean that the following groups would be put aside forever allocated for that purpose and that purpose alone. The first one and the biggest was for large corporations that needed a lot of IP addresses. And that's 10.x.x.x. So you lock down the first octet to 10 and all the rest are free for alls. Next 172. And the next number is a range from 16 to 31.x.x. And when I say .x, an x means any number from zero to 255. Okay. And then finally, the one I've already mentioned, which is the one that most people have seen. And that's 192.168.X.X. Of which most home networks are either, you know, 192.168.0. So whatever. OK. So that way, it would mean that you within your home network could have a computer called 192.168.0.2 and I could have my device could be 192.168.0.2 but it sounds crazy. How is that possible? Right? It sounds crazy, but it actually does work because of the routing. So let's take that. That address is always resolved locally. Yes, exactly. So let's take the example of two computers A and B. They each have the same address, the one I just mentioned, but they're connected via their respective routers via the same gateway IP address on their local network. So 192.168.0.1, let's say. But they are connected to the internet. You're on your side of the world, I'm on my side of the world. Router A has a public IP address A, router B has its public IP address B, and they are different because they're in different parts of the world. Now let's say they're both talking to a server and that server has IP address C, just because I'm got to keep using letters in the alphabet. Okay, so PCA and PCB have the same local IP address, but they ask server C, which has its own public facing IP address for a different web page, but on the same website served from the same server. So that server knows that each packet came from a request via a router at a different IP address, either IP address A or IP address B. That's all it cares about. And it sends its response to that respective router. That each respective router then knows which PC on its network actually originated that request and forwards the response onto that respective PC. And therefore you have ended up with what is referred to in the lingo apparently, as one to many NAT, network address translation. I just always refer to that as network address translation, you know, or NAT, N-A-T. but it's correct full name is one to many NAT. And that is the method by which subnets allow you to have multiple identical IP addresses all around the world on intranets. And yet you can still connect each of those through gateway slash routers. I mean, the router is the gateway. That's why it's kind of confusing. Some people say, well, is it a router? Is it a gateway? Well, it's essentially, it's a gateway router. It's just, you know, it performs the function. Anyhow, the point is that that's how it's possible and it works. It sounds crazy, but when you think through it actually does work. Okay, so far so cool. All right. Now that we understand the basics of routing and network address translation and how it's possible for packets to do that. Now, let's talk quickly about firewalls and then we'll talk about ports. And not the kind of port you drink. Firewalls! Obviously this is an alcoholic podcast. Although it would be a very different podcast if I drank alcohol beforehand. And all episodes of Pragmatic have been recorded sober. Just so you know. Sorry to disappoint. I'm not sure if that is disappointing. I don't know. Anyway. Okay. What is a firewall? A firewall is an access control mechanism or device and that prevents access to one or more devices on one network from other devices on either that network or an interconnected network. It's an access control device, right? It's kind of like the stop-go person on the roadside saying, "Yeah, sure, I'm going to let your message through" or "No, get lost, you can't." you know, you shall not pass, you know, that's it. And then one falls to the bottom of a mountain with a big fire breathing thing and guy got a ring and they throw it in Mount Doom and blah, blah. Okay, two types of firewall, hardware and software. So routers essentially through network address translation create essentially a hardware firewall. And I say hardware, there's no such thing really as a hardware firewall. Technically it's handled by firmware. If you prefer firmware running on hardware, which is not hardware, it's frustrating. Anyhow, nevermind that, don't wanna quibble about that definition of where that demarcation line lies. Let's just call it a hardware firewall and move on. Any requests sent to them that isn't originating from the intranet gets rejected. Unless of course you enable specific port forwarding to forward all requests to a specific port to a specific IP address on the other side of the firewall. That particular methodology, opening up a port, sometimes people call it port forwarding, can send all traffic that's sent to the public facing IP address on a gateway through to a single IP address, a single computer or device, and they'll refer to that as a demilitarized zone, DMZ, depending on how you want to pronounce it, depending on what country you come from. And DMZs have their place, but honestly, I don't think it's really considered to be the best method to handle things. Sometimes it is the only way to handle things, but generally speaking, you know, most people at home will never use a DMZ, will never set one up. So for that reason, I'm not going to go any further with it because it doesn't apply to most users. Port forwarding can still be useful for other reasons though, even if you're not using a DMZ. Like for example, let's say a while back, we talked about dynamic DNS, you got a dynamic IP address, or maybe you're lucky enough to have a static IP address, lucky, rich, whatever, I don't care. Maybe you got a way of connecting to the public IP address at your home, a wifi modem router, whatever it might be, connecting to a computer sitting behind the network, hiding there happily, protected relatively from the internet and all the dangers lurking. So you want to connect VNC to that. Well, if you just connect up most modems, it's not going to work. You need to actually forward on the correct ports for VNC traffic in order for it to find that computer. When you do so, you have to do it to that specific computer. Because otherwise, how else does the router know where to route the traffic to? Unless the traffic originates from inside the network, how would it know? You can't just say, here, send it to computer that one. It's like, yeah, which one? That one? No, no, no, no, no. Seriously, seriously, which one? That the one one? No, seriously, no. So anyway, nothing about that software firewalls. Now, they screen traffic, potentially a lot of different layers of the application stack operating system level, application level, various levels in between, or whatever. But let's take the example of OS X. Why? Because, well, I just can. Anyway, there's something called the IPFW. And that actually came with OS X as part of FreeBSD. And I'm sorry, that IPFW stands for Internet Protocol Firewall. And that there's a second firewall in OS X that they refer to as the application firewall. Now, both of these can block TCP/IP layer requests from specific ports or, or addresses, et cetera, whatever you like, all sorts of things, some more configurable than others. It's beyond the scope of this discussion, but let's just say that at an individual device level, you can also have a firewall. Windows has a firewall as well. Introduced in Windows XP, if I remember correctly, maybe Windows 2000, I think may have had it. Anyway, so that's just briefly about firewalls. Access control devices to screen traffic from coming into your device. So let's say you're on the outside and you go and you're trying to access someone else's computer from behind a firewall. All you'll get is, you know, there is no, there is not, you'll eat static. You'll get no response or you'll get, you know, destination host is unreachable or something like that. Yeah. Maybe you'll get a ha ha. Thanks for trying to get bent. I don't know. Whatever. It just won't work. OK, good, lovely. Ports, ports, ports. People talk about ports. And well, because I mean, well, why not? Anyway, IP addresses on a machine may well identify it as a unique device, the IP layer, but it's actually kind of handy to further subdivide the messaging into a virtual series of virtual constructs that's referred to as ports. Now, I don't really see the difference, not really, but between calling something a port and calling something, you know, like adding an extra dot and adding an extra octet at the end of an IP address. I don't really see the difference. I mean, okay, it's not an octet actually, but it's more than that. It's bigger than an octet. But the point is, and if I recall correctly, it's a word. So it's a 16 bit number. But the point is that ports are, yeah, essentially a virtual construct. It's supposedly analogous to physical ports on an Ethernet switch or on a computer like serial ports and so on. Like data from this thing comes through port one, this comes through port two, and so on and so forth, but really they're completely virtual. They're not real, but people talk about ports. They are still important. So think of it like an IP address is the street and the port is the house number. Think of it that way, if it helps. I mean, it may not help, but if it does help, there you go, think of it that way. So some common ports that you will have heard of, port 80 or port 8080, that is 8,080. And that's for web HTTP requests. Other common ones like FTP is port 21, HTTPS, which is secure HTTP, which we'll talk about later. That's 443, SMTP is port 25. There's lots and lots more. And there's actually a link in the show notes if you really care what those ports are, like the ones for VNC, if you really care, look it up. But essentially, yeah, those are ports. So I'm talking about port forwarding, I'm talking about all FTP requests. Let's say you would forward port 21 to this IP address and it would then put forward all requests from the internet to your public facing IP address to whatever private IP address on the other side of the firewall for that port and that port only. Or forward all your damn ports to the DMZ and watch it burn, whatever. Okay. You know, I reckon about halfway through. - All right. - How long are we getting going? It's a short show. Gotta keep trucking. DHCP versus fixed IP. Gotta mention it because well, you just gotta mention it. So fixed IP in the bad old days, man, IP addresses, they were sent by jumpers, you know? And if you're an Aussie or I think actually the British also use the word jumper, sweater. It's not a sweater, it's not a cardigan. No, I mean jumpers as in, you know, little physical jumpers, like a little shorting bar, if you like. Or if you're more modern, because it's nothing worse than changing a jumper inside a computer and then dropping the jumper. And it's like, Oh, man, where did it go? Oh, damn it got got jammed up in a fan, cooling fan. Whoops. Well, that's making a cool noise going bing bing bing bing bing. I speak from experience. Anyway, DIP switches are better, but they still sucked. Let's face it. No one likes jumpers. No one likes DIP switches. They suck. As I said previously, some Control System hardware I still deal with every day still uses jumpers and DIP switches, and it sucks. Anyway, thankfully, in 1993, the Dynamic Host Configuration Protocol, DHCP, came about whereby IP addresses can be allocated dynamically when required and then reallocated to other devices when they were no longer in use. So DHCP or the DHCP server technically is usually controlled these days by the router. I mean, it doesn't have to be but yeah, so technically, you could actually have a modem, a separate modem, a separate router, and you could have a separate server and you could have the modem just do the do the connection to the internet. And you've handled your fixed IP and you can have a Wi-Fi router handling all of the routing and you can have this server handling the DHCP if you want it I suppose but the far more common use case is you get this crummy box from your damn telco or you know whoever it is your ISP and I say here plug this in and you won't have any problems at all thumbs up and you'll plug it in and it'll do a DHCP for you and your Wi-Fi and ADSL or your Fios or whatever you've got and it'll just you know still be a headache. Anyway so on the device side it's settings like obtain your IP address automatically in Windows LAN, slightly different in OS X it just means it's using DHCP. So the benefit of using DHCP is you can never accidentally define the same IP address twice on the same network which would be bad because then you would get duplicate packets, packet collisions, lost data and you would be very sad. And we're trying to avoid sadness. Anyhow DHCP though can give you more than just the IP address. It can also give you a whole bunch of other network information which we will talk about a little bit further on. And you want to know more about how the ins and outs specifically of how DHCP works then there are some links in the show notes for further reading if you really are interested. Okay, DME, we're moving our way up the stack. But before we work our way too much further up the stack, let's talk a little bit about, not too much, but a little bit, about ISP, inter-ISP communications. - Okay. - So up until now, we've been talking about local intranets. we kind of briefly touched on the fact that packets go out to a server on the internet and our ISP is connected to the internet. But what the hell is the internet exactly? The internet is really just a bunch of switches connected together. There's really not much else to it from a theoretical point of view. Practically speaking though, ISPs talk, have to shuffle the data around somehow and how they do that, they do that via backbones. or what we refer to as backbones. Now, traditionally, backbones were all circuit switched data. Now I've talked about circuit switched data previously, and the idea is packet switched versus circuit switched is that, you know, if I'm talking on a phone, I pick up the phone on one end, I get a time slot in a digital backbone, and that time slot will be, you know, I get this channel on this T1, and it's assigned to my phone call 64 kilobits per second, and it's reserved for me. and that circuit is switched all the way through the network. No matter which telephone exchange I go through, I get allocated a slot. Once I got a slot allocated on all of my connections to the destination, the destination will ring. Ring ring. Got it. Yo, Vic, what's up? Nothin's up. Nothin's up. There you go. Thanks. Bye. And I hang up. And then those circuits are then put back into the pool, and someone else can use them. But during the duration of that call, that circuit is ours, right? One up, one down, fixed rate, always ours, whether we talk or whether we don't. Horrendously inefficient, because when you think about it, a lot of this time you speak is not talking. It's pausing. And I mean, let's face it. That's why Marco Armit's SmartSpeed works so well, because on podcasts and an overcast, no, they're not a sponsor. Yes, it is a good app. And well, I mean, smart speed is is awesome. You get addicted to smart speed. But I'm just saying, and then you listen to them talk live and they're like, geez, they're talking slow. Anyway, never mind that. Thanks, Marco for that unintended side effect of your... Anyway, so not talking about podcasting apps. But the point I'm trying to make is that the bearers, the backbones, all used to be circuit switched. And eventually there were better methods determined. So some of them, the technology of the carrier itself, like the fiber optics, the sort of fiber optic line drivers and receivers that I came across, what they will do is they would take all of the data in different formats, be circuit switched data, packet switched data, and they would essentially combine them into essentially a wavelength division multiplexed fiber optic transceiver. That's a big mouthful. The name of the product I came across when I was working at Nortel wasn't part of the wireless division, but our equipment connected into it was something called OC192. There's a link in the show notes if you're interested. It was cutting edge at the time. It was a 10 gigabit SDMS WDM fiber optic interface. And that could handle 16 different wavelengths per fiber core. That was, you know, cutting edge at the time. And you start to get into wavelength interference and all sorts of other problems after that and blah, blah, blah, blah. But anyway, that was 1997 cutting edge stuff. It supported some SONNET, SDH, which is, sorry, SONNET is synchronous optical network and SDH is synchronous digital hierarchy. And that formed, so SONNET is the basis for POS, which is packet over SONNET. Jeez, I hate it when they do that. You know, it's like there's an acronym inside an acronym. So it's like POS as opposed to POSONET which I guess POSONET or POSONET doesn't really work so I guess it's POS. But POS also stands for Point of Sale so anyway whatever, blame Cisco. Cisco played a big role in development and pushing that to the market for POS Most internet traffic carrying over backbones uses POS in one form or another I mean it's much more complicated than that technically technically, but essentially, imagine your traffic, your packets that you're sending and trying to receive and a million other packets being routed over a subsea cable between the continents, extracted and spanned and switched out at the other end by a bunch of ethernet switches and routers. The routes can be learned by the broadcast method or they can be manually configured for load sharing or balancing. So you can say, I want all IP addresses from this main hub to this main hub to go via this link or this other link, or you can just let them learn either way, link goes down, I don't want a redundant link. Anyway, all of this load sharing, load balancing, clustering, how ISPs balance their traffic loads and all that, it's actually really fascinating, but for the average individual, suffice to say, yes, it's done, yes, it exists, no, I'm not gonna go any more detail about it because I don't think it's interesting enough. Well, I do think it's interesting, but not interesting enough, so for this discussion. So there's a really interesting article that I've linked to on the show, it's called "Internet Cost Structures and Interconnection Agreements." And it's a really fascinating historical progression of how ISPs have evolved with connection agreements from the circuit switch days, through ATM frame relay and all of the packet over sonnet and all the different technologies have developed and evolved into how they've evolved their connection agreements, their line leasing contracts, and how all that's passed on to us, the end consumers of their services, such that we just pay a fixed rate. And of course, some people see the raw figures and say, but that's ridiculous, they're ripping us off. And it's like, yeah, well, the last mile is quite expensive to maintain. Plus, you gotta deal with people. And that's always expensive, because people complain all the time. Like me. Anyway. Okay, it happens. It happens a lot, indeed. All right. D-N-S. Dns. Dns is our friend, because we human beings don't like numbers because our brains don't like numbers, unless you're some kind of mathematical genius, but... Or you've got an eidetic memory, which is... Never mind that. Okay, so IP addresses are all well and lovely and a beautiful sight to behold. Actually, no, they're not. But how the heck is it possible for me to remember them? Because the human brain doesn't remember numbers. We just don't. We sit there and we recite phone numbers. I can recite my phone number from when I was a child because I had to learn it when I was at school. In fact, I can actually remember most of my phone numbers from most of the places I've lived all around the world because I just got a weird memory for numbers, but I accept the fact that I'm weird and most people don't. And Frank, I can't remember IP addresses apart from 127.0.0.1, because, you know, let's face it, I got scammed once and now I'm forever angry. But the point is that they had to come up with some way of dealing with this problem. We say visitgoogle.com, or maybe we say don't visit Google, but either way, we say visitwhatever.com. We don't say visit 127.27.27.17. We don't. And there's a lot of sevens in that. - That's catchy right there. - Damn straight it is, man. That's the catchiest IP address ever. I am totally registering that. So how do you get around this? The answer was DNS. The domain name system, sometimes referred to as domain name servers, DNS, same abbreviation, subtly different meaning, solves this problem. Back when I started at uni, DNS was not a big thing. Certainly a lot of sites had them, a lot of institutions had them, but there was still a lot of servers, especially the good ones, if you know what I mean. They were just IP addresses. And once you knew the IP addresses, you wrote them down and you kept them somewhere safe. Because, you know, they didn't register with the DNS for various reasons. Anyhow, and a lot of the protocols, I guess, we're using sort of at the time, you know, like IRC or Telnet even. You know, it's just raw IP addresses were okay. It was early days, but it was hard to remember. So the best way, easiest way to think about DNS is like a phone book for IP addresses. Now, the thing is though, ARPANET actually had a file, a text file called hosts.txt and that kept that information, but it really wasn't that useful. It wasn't extendable, wasn't easily searchable, didn't handle a whole bunch of edge cases, so essentially it was abandoned. The original DNS specification was actually written in 1983, so pretty early on, but it was greatly extended in terms of its usefulness in 1987. In fact, the most common DNS server software in use is called BIND, and BIND is the Berkeley Internet Name Domain. That That was actually written originally in 1995 for Unix, but it was ported to Windows NT in 1991. And today it's still the most popular used DNS server software. It is a niche application, I'll admit. So there's probably not much competitive benefit for other people to go and develop a competing platform or product, but anyway. So what the hell is a domain? Everyone knows what a domain is. Maybe they don't know why it's called what it is. And in fact, one of the sponsors of Pragmatic, Hover is a domain registrar, which we'll talk about in a minute. They're not sponsoring this episode by the way, but still. Domain names are essentially what we would say, google.com or www.google.com or .com.au or .co.nz or whatever. Those are all examples of domain. Domains, techdistortion.com, that's the domain of my site. So domain names are broken down into segments. Sometimes they're just recalled parts and more often they are formally referred to as labels. They're separated by full stops. So domains are read backwards. And when I say backwards, I don't mean you read them like "throughout the virtual radar system" and not. I mean, anyway, I mean, you read them from right to left as opposed to left to right. And I mean backwards only from a Western, predominantly Western writing point of view, because it's forwards, you know, when you think about it from an Arabic or Hebrew point of view. And actually, if you're interested, there's a link in the show notes about all the languages in the world that are actually read the other way around. So there you go. Anyhow, that's if you're really, really interested. I'm sure someone is. Okay. So the specific rightmost domain is referred to as a TLD, top level domain. Common TLDs include .com, .net, .fm, and most importantly, .coffee. Anyway, country codes, yeah, well, you know, country codes are also appended on the far right. And although you can argue that technically, these are actually the top level domain, they are not usually referred to as the top level domain. They're usually referred to just as the country or the country domain. Or sometimes they're combined with what's referred to as the TLD and it's combined is called a TLD. So for example, .com.au or .com.nz for New Zealand, .au being Australia, of course, .nz New Zealand or .com or .co.uk for example if you're in the United Kingdom. So these country codes plus their TLD can sometimes be referred to as the TLD. So it's kind of funny. But the weird thing is that .com started out very early on as a lot of the domain stuff is and you guessed it, you've got ARPANET, you've got Berkeley, where's all that stuff? It's all in the States. lot of this stuff started in America therefore dot-com typically does not have dot-us on the end of it in fact I don't think I've ever seen dot-com dot-us despite the fact that dot-us is the United States actual country code it's hardly ever used some say it's implicit but it's also it's also not genuine because tech distortion is tech distortion dot-com it's you know technically it's hot it's it's dot-com dot-com doesn't tell you what country it's from. Therefore, it's implicitly from the US, but I'm in Australia. Annoying as hell, isn't it? And let's be honest as well. What's ATP.FM? Relay.FM. FM is French Micronesia. Neither of those are actually in French Micronesia. OK. FM just sounds cool because it sounds like a FM radio. Frequency modulation, FM. Y'all, baby. But here's the thing, right? You don't have to live in the country where that actual TLD country, you know, code, whatever you want to call it. You don't actually have to live in that country, but in order to have that. But the registrar from it has to be, I believe, have access to the owner who is in that country in order to use that domain, if I remember correctly. Anyway, I'm sure someone will correct me if I'm wrong about that. Irrespective. So that's the TLDs, but the actual domain name, the thing that we think of as the domain name is the next component to the left of that. So for example, Apple's is Apple. So you've got apple.com. So Apple is their domain. Google's is google.com. So, you know, what's left of the.com? Google. That's the domain. I think I see a pattern here. Are you seeing the pattern? Are you getting it? I'm getting it. It's three devices. Are you getting it? These are not three devices. Anyway, fantastic. Anyway. Okay. Now beyond this, sometimes you'll see the domain broken down even further. And that's generally either by a sub domain or I guess what I like to call a service type or a service class. So sub domains can be for subject specific or service specific purposes. So anything you want really, cause there's no rules, but for a while, going back a year and a half ago, So before, actually, yeah, no, it was before Pragmatic went indie. I had podcast.techdistortion.com. So podcast was actually a subdomain underneath tech distortion. And I had a separate, a completely separate WordPress installation, completely separate WordPress everything. Because I was managing the podcast portion of the website that used to host existential. That was done under its own WordPress installation, its own website. and it was hosted under a sub-directory on the same server. So the same server was actually hosting two different websites. You know, not a big deal. Technically, it's just that's the way I chose to do it at the time. Eventually, I had a divergence. I do work on like the theme files on one site. I'm like, oh, I'd really like this available on the other site. Then I had to port it from one to the other and it became a pan the neck with code bases. And I'm like, grumble, grumble. I'm a developer, grumble, grumble. I've had enough. Spit the dummy, move to Statomic, and then I combined the two and I got rid of my subdomain podcast.techdistortion.com. When you buy a domain, you can usually then register as many subdomains as you want, usually, it's up to you. So, I also mentioned, however, a service type. So for example, let's say you've got a service like an FTP server or a web server. Well, if you had an FTP server, you might say ftp.techdistortion.com, or if it's a web server, you might say www for World Wide Web, .techdistortion.com. But in most cases these days, it's all about the web and domains are about the web. Therefore you say techdistortion.com. You don't have the www in front because there is no other service associated with techdistortion.com. There just isn't. Okay? Make sense? - Got it. - Okay. So, Finally now we are at URLs, a Uniform Reference Locator, which is essentially the name of the website, its domain. And the purpose of that domain is to make it memorable. So I say to you, "Visit techdistortion.com." And then you're like, "Is that one word? Is that two words? Is there a dash in the middle? Is there an underscore? Is it uppercase, lowercase, uppercase D, lowercase D? Is it tech as in T-E-C or T-E-C-H? Or does he mean technology? Or yeah, it's like, so every now and then, I do regret my choice of tech distortion, but oh well, most people get it right. But the problem is of course, all the common words are long gone. They were long gone a long time ago and they get a lot of money. So people will have this thing where they'll sit on a domain name and I'll say, I'm sitting on coffee.coffee. Why? 'Cause I'm a pain in the ass. I'm never gonna ever brew a cup of coffee, but someone else is, and they're gonna want coffee.coffee, and they can pay me $200 million for it. So-- - Good luck with that. (laughing) - Indeed. So yeah, every now and then, these guys get lucky, and someone, a big corporation really, really wants it, and so they'll pay stupid amounts of money for it, because it matters. I remember Apple paid a lot of money, I think it was for me.com, and iCloud.com was another one, I think I paid a fair bit for, I think Marco may have paid a reasonable amount for overcast.fm, I can't remember. I know he paid a lot for the trademark, but you know, people pay money for these things because it's all about the brand. Anyhow, so techdistortion.com, there you go. That is a URL. Technically, you've got to add http://www.techdistortion.com/ for that to be a complete URL, but that's another story for later in the podcast. We'll get to it. Okay. All of this, all of that BS. Just so I don't have to remember 127.34.10.7. So DNS's, domain name service does more than just hold that information. it does a whole lot more. As I said, it was extended in 1987. So they added a whole bunch of really cool stuff. They store more than just the IP address, but I'm not gonna talk about everything that they store or can store. There's a link in the show notes if you wanna know every single thing that they store, but here's the thing. We'll talk about the key ones, but rather let's talk about the ones that I use. So first of all, A or four A's in a row, quadruple A if you'd like, A, A, A, A. So A or AAAA, A. IPv4 and IPv6 respectively. And that's a domain association. Essentially, it's the most important one. It was the reason for being initially, at least for DNS. And it returns the IP address. So you type in the domain and you associate, your A record is associated with the IP address of your server, which is hopefully a fixed IP. You're in trouble if it's not. So that one's the one you'd want. CNAME stands for canonical name record. All that is, is an alias of one name to another. So you can say Bob and the alias of Bob is Bobby McBob. I don't know, whatever. MX, mail exchange record. People have heard of these, I'm sure, if they've ever done any mail stuff, mail service and so on. A mail exchange record maps a domain to a main mail transfer agent for that domain. But the other thing about MX records is you can set a priority and this allows you to have backup mail servers. So what you can do is you can have like two, three, four of these so that if one mail server dies, you have a backup mail server 'cause mail servers can be load balanced, load shared, you know, redundancy and all that rubbish. So, you know, not wanna talking to, not gonna be talking much about email in this episode, frankly, because it bores me to tears, but that's okay. Anyhow, suffice to say MX records, that's what they do. TXT, TXT as opposed to TNT is a text record. Now, it doesn't actually do anything but store textual information. Well, that's probably not all that surprising actually, come to think of it. More recently, it's been used as a method of determining ownership of a domain. So for example, when I changed from a dodgy domain registrar to an awesome one, being Hover, I moved across my Office 365 exchange entries for example because I actually I subscribe to Office 365 and I use their exchange service and my tech distortion.com email is actually hosted Office 365. No, Microsoft are not a sponsor. Anyhow, you'll enter a TXT entry to prove to Microsoft that you actually own the domain in question. After the move is completed that text entry serves no real purpose. All they're trying to do is to stop other people stealing and redirecting your mail. That's all. And the text entry provides a mechanism to prove, yes, you have right privileges to that DNS record. So it's safe to know that you are the owner of that domain. Okay, last one I'm going to mention is SRV, short for Service Locator. Now, it's simply a more modern and frankly, far more multi-purpose entry than simple MX records. Office 365, for example, uses these for links, communicator, and a bunch of other stuff. So SRV entries are useful for that. Okay. So these records are held by so-called root name servers. Some people just call them on name servers. - Yeah. - And when a request goes out to a DNS root name server, the query is recursive and it's recursive on the essentially in the order of the domain. So the .au, the .com, the .tech distortion, the .podcast, .www. So they are essentially are recursive through a series of tables to the point at which they resolve hopefully to an IP address that relates to that domain. and that data is then returned such that your computers can then say, right, I've looked you up in the phone book. Now I know your IP address. Let's go. Give me data. So because all of this can be time consuming, DNS records are usually very heavily cached because let's face it, IP addresses really don't change very often. So admins. Yeah, exactly. So admins running sites will specify different TTL. That's time to live, not transistor to transistor logic to push updated IP addresses and namespace records far more quickly. But honestly, even with that, there can still be big lags in different DNS servers around the world because their caching rates may not observe the TTL for a bunch of different reasons. Traditionally, you get quite a 24 to 48 hours maximum. However, the reality in my experience the last 12 months when I've been changing servers, I've done I've done that a fair few times, last few, I think I've done it three times in the last 12 months with the podcast and Statomic and changing to Cloud Shards, for example, and DigitalOcean and all that. It really only took an hour or two and it was fully propagated. And there's a tool available on the web called the Global DNS Propagation Checker. There's a link in the show notes and I've used it regularly. There are others, it's just I used it so you might find it useful. It's not necessarily a recommendation, but it works. Okay, it's free for what I want to do anyway, it's free. And you can check to see how your DNS records are propagating around the world with key root name servers in their list, which has your pretty map of the world and a big red X or a big green tick at each of those locations where a root name server is located. So how do I get me one of these domains? The answer is you go and speak to a registrar, as I mentioned before, just like a previous sponsor of Pragmatic, Hover. And Hover is of course, one of the best in the business. Now you pay an annual fee or even two or three or four or five year, if you'd like to reserve it for that long, and you can reserve that domain name. And then at that point, you can choose where to host the DMS and which name service to use and so on. Now, in my personal case, I use DNS at Hover, but for a while I actually pointed the DNS at Hover to the name servers at DigitalOcean. And I then managed all of my DNS entries from DigitalOcean. I had my reasons for doing that at the time, but now I'm no longer with DigitalOcean, I decided to simply move all of my DNS entries and let them be handled at Hover instead. And it works fine. So anyway. - That's who I use. - Yes, exactly, Hover Rocks. So, okay. So, right. One of the interesting points to note though, is that some people are saying that domains are dying, that domains have had their day. And there's a couple of reasons why people are saying that. There was a whole bunch of TLDs that were released about 12, 18 months ago, things like .coffee, for example, and .business, I think, and .office, or I don't know, there's a whole bunch of ones, .crazy stuff. Anyhow, those TLDs will help. But the truth is that those TLDs don't change the fact that search and search engines have, and social media as well, which is a different form of search, it's a form of selective curation, that has partly removed, not completely, but partly removed the need for the absolute need for a domain. So a lot of people, like for example, - On in this episode, in every episode of Pragmatic at the end, I tell people about the Pragmatic Show Twitter account. But there hasn't been a link in the individual show notes for that until recently. And even then it only appears in the show notes if you have the full show notes. If you go into iTunes, it doesn't appear in the iTunes show notes and there's a whole bunch of reasons behind that and you know, whatever. It doesn't really matter that much because most people can type in, you know, Pragmatic Show into Google and say Pragmatic Show Twitter and it'll come up with the correct link. So suddenly I don't need to say twitter.com/pragmaticshow or one word or lowercase. I don't have to say that. People don't have to write down, people don't have to remember it. You just type it into Google or you type it into DuckDuckGo or your search engine of choice, Bing, whatever. Point is that do you really need DNS? If you wanna know about me, you can just type in John Chidjy into Google and it will tell you and it'll bring up tech distortion. It may not be the first, maybe it's the third, fourth, fifth, sixth, seventh entry. Don't know, probably doesn't matter. Either way, you know, it's just, yeah. So anyhow, so what's the point? What's the point I'm getting at here? I'm getting at DNS is less, domains are less critical today than they have been in the past because search is so good. That said, I still think it's absolutely critical if you have a business online and you want to be taken seriously, that you have a domain. I honestly, I still genuinely believe that. I hesitate to recommend to anyone that you rely on a third party service for people to find your service. You know, if your solution is go search Google to find us, when Google goes in the tank, you got nothing. Now, I'm not saying Google's going to go in the tank, but if you rely on third-party services, see DNS is a standard. It's handled by multiple registrars distributed around the world. There are multiple root domain servers, all our name servers all around the world. It's heavily redundant, highly specific, highly specified, and it works well. Well, it works well enough. So why would you then pan all of that power over to a search engine owned by a company And that company has proprietary search technology and could go on to tomorrow because Larry Page and Sergey Brin decide to, you know, go on that flight to Mars and never come back. I don't know. Or better than that, build their own planet and go and move there. 'Cause they got that much damn money. They don't know what to do with. So they'll build a planet, planet building, you know, like slightly but fast. Okay. So bottom line is, Magrathea, I should say was the planet. Slutty barf, I was just this guy, you know. Bottom line though is that DNS servers themselves are just servers, okay? They're a device at a specific IP address. So that begs the question, how does your PC know where to find the phone book? It's just, it's in the kitchen under the sink. Go look. And if that wasn't funny- - I'll be right back. (laughing) And if that wasn't funny, it's because it wasn't funny. So how does it know? Well, I said before, if you recall, not so long ago, although it may seem like an eternity at this point, who can say, I said it was gonna be a short show, using, having values passed to it by DHCP. So when you get allocated the IP address, you can pass on that information as well, potentially. Don't have to, but you can. So then how does the router, let's assume it's your router that's doing your DHCP. How does it know? Well, that's simple. When it connects up to the internet service provider saying it's a wifi modem router, let's say it'll connect up and it'll say, hello, Mr. ISP or Mrs. ISP, not wishing to be gender specific since it's actually not alive, it doesn't matter. So why am I obsessing about it? Anyhow, the ISP will say, we have a DNS server and it is the best in the whole world ever. And it's like, sure, I believe you. Anyway, and we'll just use them. - 'Cause they never lie. Yeah, they never lie. Totally solid, solid on every level. Anyhow, so it'll just take their DNS server settings will be passed to your router, and then to your computer or your device, whatever it might be iPad, smartphone, Android tablet doesn't matter, you know, yeah, it'll be passed on. So that's how you know the IP address of the DNS servers. Once you've got that information, of course, you can do a DNS lookup, it'll get an IP address. And then of course, you're off at the races as they say. However, let's just say you don't want to do that. You say, I don't like my ISP, although I'm forced to use them for reasons that I will not go into. But yes, I want to use my own DNS server. When I say my own, I mean, I want to choose my own. Well, Google, amongst other things, being good at search, have their own DNS servers that are rumored to be rather quickish, regularly updated, and just, you know, shiny and generally sexier, Depends on which geek you talk to. No kidding, whatever. Google DNS. What else are they going to call it? Their primary IPv4 is famously, famously, I don't know. I know this. I wonder how many people know this. Anyway, 8.8.8.8. 8.8.8.8. The secondary is 8.8.4.4. And yes, DNS servers typically come in pairs, a primary and a secondary. Such that if the primary DNS is unavailable, the secondary, which is a complete copy of the primary, is there in case of outage, failure, whatever. Now, the IPv6 addresses are way too long. So who cares? If you really care, go look it up. There's a link in the show notes if you're interested. Okay, so that's DNS. Okay, we are building up our layers. Vic, you still with us? - I am. - Okay, fine, good. - You haven't lost me yet. Good, that's a start. Now, I don't know how the listeners are going. I hope they're in the same kind of place. Oh, they're 3xing us by now. Yeah, they're either 3xing, they stopped or they've given up or they're just loving it. Or they're saying, "When's he going to get to Gopher?" Never talking about Gopher. Or Archie, not talking about Archie. Okay, HTTP. Hello there to people. Now that's not what it stands for hypertext transfer protocols, what it stands for. And that is the primary internet protocol that I am going to discuss, I am not going to talk about any of the others at the application layer, because let's face it, pretty much everyone on the internet uses HTTP at some point where they realize it or not, and only a handful use IRC, FTP whatever so I'm only going to talk about HTTP. If that is a problem, I don't care. There you go. Fully first fully documented standard version 1.0 appeared in 1996 with 1.1's official release, there was a draft released not long after that, but the official one wasn't released till 1999. As it stands today, technically HTTP 1.1 is what we are still using. However, that company we keep on going on about, Google, them, they came up with this, they designed this thing called Speedy. And it's not speedy, think of speedy without any vowels in it. And you pretty much got it. SPDY. Speedy, the vowel-less speedy. I wonder how it could be speedy without vowels. But anyhow. Okay, so I think, and some people say it was totally developed as a response to Speedy and it's like, well, maybe Speedy kind of gave them a kick in the pants, I guess. Is that the definition of being motivated? I suppose. Depends on how hard they kick and what kind of boot. If it's a steel cap, definitely. Anyway, the IESG and I'm sick of explaining what IESG and every other damn thing stands for. So you can look that one up yourself. IESG approved. HTTP originally was called 2.0. but then I just said, nah, dot points are so 90s. We're just going to call it HTTP2. And I have now, as of last month, February, 2015, if you listen to this in the future, when I say next month, last month, sorry, you'll know what I'm talking about. It is a proposed- - They might still be playing it then. - They might. (laughs) Oh, that's good. It's been published as a proposed standard. So it is still not technically a released standard yet. It could be months, it could be years, but therefore I'm not talking about HTTP 1 versus 1.1 versus 1 versus 2.0 or what are you gonna call it? I'm not talking about that, otherwise we're gonna be dead before I finish. Instead, we're just gonna focus on HTTP 1.1, what it is and how it works in a nutshell, okay? in a nutshell. And this is me in a nutshell, help them in a nutshell. There you go. Austin Powers reference moving on. When you load a web page in a browser, it is called a HTTP session. A HTTP session is really a series of network request responses. The HTTP client, which is a web browser in pretty much, in pretty much every case, most cases, I guess it's safer to say most cases. will initiate the request by setting up a TCP, which is Transmission Control Protocol connection, talked about that previously, to a particular port on the server. Ports, oh, we talked about those. Typically port 80, occasionally port 8080. Now a server running a web server application is set up to listen on that port or ports and waits for a client request message. So it's sitting there listening, saying somebody, somebody asked me for something, please, please hit me up for something, thank you. Upon receiving said request from the client, the server will send back a status line, something like HTTP/1.1200, okay, something like that, with a message with it. Hopefully it's not 404 not found or something like that. That would be bad, or at least not useful, or not constructive, or not what you're looking for. Unless you are looking for the 404 page, because some of them are hilarious. Anyway, the top three server applications in the world, I know I said, I mean, web server applications in the world at the moment are, can you guess what they are actually, Vic, web server applications, top three. - Well, I know Apache, 'cause that's the one I played with. - And that is certainly one. What about, what's another one? Honestly, I don't know. I don't pay a lot of attention to this. You're a code monkey. Yeah. You just keep falling back on that one. Fine. Okay. Then Nginx. That's the one I use. And a lot of people use Nginx rocks. It is very, very cool. And of course, Microsoft IIS. And if you don't know what IIS stands for, that's internet information services. And regrettably, I did not have to look that up. Anyway. So, just out of interest, if anyone is interested and if anyone's still listening, Tech Distortion used to run on Apache. But when I brought Pragmatic over to the site when I went indie, I had to move to Nginx basically 'cause Apache kept dying. Nginx handled the high traffic. Yeah, oh yeah. Nginx performs far better than Apache for surges in spikes in traffic, such that when the episode goes live, there's a massive surge of download requests and page requests. So when that happens, Nginx responds quickly enough on a very light amount of server power, whereas Apache just falls over. So I tried tweaking Apache, I tried everything that I could think of that I read about, and it just did not fix it. Switching to Nginx fixed it. - Interesting. I thought it had a rock solid reputation. - Apache is rock solid. It just can't handle that sort of load. Some people prefer Apache for a whole bunch of different reasons. And what they'll do instead is they'll have, uh, they'll do server load sharing. They'll do, you know, they'll have more powerful servers, maybe instead of being a cheapskate like me and running on a $5 digital ocean server or even better, a, uh, an $18 a year VPS at CloudShards. No, they don't sponsor the show, but they probably should anyhow. and I love my CloudShards VPS. It's running it now and it has not fallen over yet. Running Nginx, oh yeah. So, da da da da da, where am I up to? Oh yes, right, good, good, good. It's easy to get lost on 12 pages of notes, believe me. So servers are typically running a flavor of Linux or Windows Server, some flavor of Windows Server, 2003, 2008, 2000, God knows what. or Linux like, you know, Red Hat or, you know, geez, you know, Ubuntu or not as common Ubuntu for running a server, but you can do anyway. All right, so, so yeah, my case, Tech Distortion is running on CentOS 6 with Nginx as its web server and Statomic is its CMS, Content Management System, which is the front end that contains all the web, the actual data. It's, Statomic is now, my Statom- Tech Distortion is now heavily cached. It used to have a lot of PHP crunching in the background. Now I've got a layer of caching in there. It's not using varnish, if you're curious. How to play, didn't think it needed it because Datamix got some pretty good caching capabilities built into it. Okay. Quickly want to talk about HTTP and HTTPS. So HTTPS is a hypertext transfer protocol secure. So you can argue that you've got secure and non-secure HTTP. So secure allows end-to-end encryption between the client and the server. For that to work, it requires a valid trusted service certificate from a certificate governing body. And we talked about this in episode 55, and I'm not going to say anything else about it. So if you really care, go have a listen. Some more of the common HTTP requests at the HTTP layer include get, head, post, put, connect, and options. There's actually more than that, but those are the common ones. And of those, the far most common ones that I've come across are get and post. So get is the most common request a web browser will put out there. It'll say, you know, get index.html from techdistortion.com. And that is all that'll get sent out. So, okay, here's a typical request, right? this is what one would look like. A typical request for the root page from techdistortion.com in HTTP would look like this. Get/index.html, HTTP/1.1. So that tells you the standard, it's a get request. This is the specific file that I'm looking for. Host would be techdistortion.com. That's it. That's the get request. That's what would go in the packet as the packet payload from my IP address to the IP address of the server that I've then determined 'cause the techdistortion.com will get resolved down to an IP address. And that'll go to the IP address of the server via the DNS will resolve that. It'll go to the IP address of the server. The server will then get that packet. It'll have a look at the message payload and it'll say, right, I have a GET request for this and so on. Who handles that? Well, it's coming on port 80. So who's listening on port 80? Nginx is listening on port 80, great. Nginx says, "Oh, it's a GET request. Okay, it's looking at this file, checks a bunch of things and says, "Right, I'm now going to package up a packet message in response to it, the GET request." So the response coming back from Nginx will look something like this, HTTP/1.1 200 OK. Date, blah, blah, blah, sometime and date, usually in GMT. Server, it'll then tell you what sort of service. It'll be like Nginx version, blah, on server operating system, blah, blah, blah, whatever. Last modified dates, blah, blah, some date and who cares? Then ETAG is actually short for the entity tag and that's an hexadecimal. It's actually used to, as a form of figuring out cache versioning and invalidation on the client side because there's a lot of caching going on. So that, you know, you can tell if the version you're getting is the same as the version that the server has. Content type in this particular request is going to be text / HTML, character set will be UTF-8 most likely. Content length, however long the message is going to be in bytes. Accept ranges in bytes, connection close and then of course followed by the actual payload of HTML itself. And I mean the HTML because sometimes people get HTTP and HTML mixed up. They're two different things. HTML is hypertext markup language, not the protocol. HTTP is the protocol, HTML is a text markup language. Now I alluded to this, I mean, that's it, right? And the web browser's job is to then display the HTML in a human readable way, okay? So, and no, I'm not talking about HTML. That is just, I'm not. So because bandwidth is so precious, and I say that like it's less precious these days, I guess so, but it has been very precious and remains precious for many people on bad quality internet connections or in places that have unreliable electric supply. And bandwidth is, yes, bandwidth is precious. So clients will rely heavily on client-side caching and that'll keep the local copies of frequently loaded web pages, images, or any other content so that you don't have to redownload it every single time the page is refreshed. And that's usually a good thing because it cuts down the time it takes. It's quicker to read it from the hard drive or solid state drive on your local device than it is to download it again off the internet. Generally speaking, that is the case. But it can be a pain in the neck if the server has an update or a correction applied to their website. And although Nginx or Apache is serving out the correct latest version, your client has cached an old version. So all caches have an expiry time. And if your cache hasn't expired yet, you won't see the update. I mean, there are manual ways to clear the cache depending on the web browser that you're using, but you know, it's that way you get the absolute latest version, you know, so it's just something to be aware of anyway. Okay, so that, in a nutshell, is from web browser on your device to the server and back again, how the networks times hundreds of millions of people and devices. It sounds like it couldn't possibly scale, it couldn't possibly work and honestly I'm sometimes I'm stunned that it does but it does work and it works very well. It's wonderful. It is a wonderful thing. So as a way to wrap this up and tie this together in this the mother of all damn pragmatic topics. I just want to try and play the scenario for you. OK, here we go. Now, I'm sitting at my desktop, my laptop, my smartphone or tablet connected by Wi-Fi, most likely to the Internet at home, and I open up a web browser. I type in Google.com to do a search. What happens? Your device is connected to the network at some point in the past. It's obtained an IP address, the DNS server settings from the DHCP server, which happens to be a Wi-Fi router, which happens to be the annoying box your ISP shoved into your hands when you signed up. But what the hell, right? Your web browser assembles a request to Google.com, sends it to the router, who, having no idea what that actually means, has no history in cache for Google.com. Go to the ISP's DNS server, does a DNS resolve to get the IP address 74.125.237.169, which by the way, that's the IP address of google.com, then forwards on the request packet requested packets to the IP to that IP address. The first stop for the packets is the ISP. And that knows that the addresses in America based on its routing history, let's say that this is coming from outside of America, you know, the UK, Europe, Asia, Australia, wherever, okay, Antarctica, who knows, and forwards it on to an ethernet switch or portal in the nearest big city wherever it's allocated to go next, which then passes that onto a backbone cable across the ocean, it comes out in America and passes through a few more switches before ending up at google.com's server at that IP address I mentioned before. I'm now ignoring the localized country redirecting any other local caching, server-side load balancing, let's all just assume it's one server with one IP address otherwise we'll never finish the damn episode but yes okay I know it's not quite that simple but that's simple enough. The google.com server opens the packet and the HTTP serving software of choice Apache, Nginx, whatever the hell Google are using and I probably should know that but I don't and I don't care at this point, responds with a series of packets and those packets of data will include images, text, maybe some PHP is going to run in the background on the server to serve up something specific to the request and the data gets sent back to the original location with the originator's IP address reverse translated back to the local network again via the router that it left through eventually. Now the web browser on your device will receive those packets and unwrap them and assemble them. It'll probably get the packets via exactly the same route but not necessarily across the internet could go a completely different way you just don't know probably in the right order but again not necessarily. Some may go some path, some may go another path, but hopefully then it'll reorder them into the correct sequence if that's required and probably you know it'll receive those packets quickly except if they live at my house in which case it'll then display all of that information on the screen of your device in accordance with the HTTP standard. And all it took was your device, a Wi-Fi modem router, your ISPs modem, the ISPs background, trans-oceanic subsea cable, ISPs, telco switches in America, Google service switches and all that infrastructure to make it work. Boom. Now have a think about all the software, all the hardware, all the firmware, all the electricities, every single standard required every single level of the TCP/IP stack, not to mention the damn web browser, just to make that work. Welcome to the internet and we're done. Excellent. What do you think? I think it's good. Okay. We made it. We did. We survived. Holy crap. We're still on the internet. Oh, my God. Well, I'd say it's time that we wrap this up. Now, before the show ends, if you're still listening, before the show ends next week will be our last show. I have one final, final, super final totally. I promise it's the final vote. Listeners can participate in if they want to. you can go to techdistortion.com/pragmatic and there'll be a link in the show notes if you don't wanna go to that domain. And you can vote on your favorite episodes of the show. It's anonymous if you want it to be. I'll be telling the results for the final episode next week. Now, as an incentive for those that are interested, I'm going to pick out three random entries with valid email addresses that will be announced during the final episode and they will each get a free sticker sent out to them wherever they may be. Sound cool? Sounds cool. Fantastic. Well, if you would like to talk more about this, you can reach me on Twitter @JohnChidji, and you can read my writing and you'll see this podcast and others that I've made in the past. They're all hosted at my site, techdistortion.com. And that is a domain, yes. If you'd like to get in touch with Vic, what's the best way for them to get in touch with you, Vic? They can find me on Twitter @vichudson1. Yes. Through the internet. through the internet. Oh yeah. And if you'd like to send any feedback, please use the feedback form on the website that's hosted using Statomic CMS on an NGINX web server, blah, blah, blah. Anyway, and that's where you'll also find the show notes for this episode under podcasts Pragmatic. You can follow Pragmatic Show on Twitter to see show announcements and other related stuff. I'd also like to say a final thank you to both of our sponsors for this episode. Firstly, to Lynda.com. If there's anything you'd like to learn about and you're looking for an easy and affordable way to learn then Lynda.com can help you out. Instantly stream thousands of courses created by experts in their fields of business, software, web development, graphic design and lots more. Visit lynda.com/pragmatic to feed your curious mind and get a free 10-day trial. There's something for everyone so if you've ever want to learn something new what are you waiting for? I'd also like to thank SapientPair and their iOS app Shopee for sponsoring Pragmatic once again. If you're going shopping and you want a great collaborative shopping list app, then Shopee can help you out. It's ad free for the first month. So why not check it out at sapient, that's sapient-pair as in two.com/pragmatic. Make sure you check them out, everybody. Thank you, everybody for listening, for voting for that and making it the most pop voted for most pop most interested in topic. And I imagine after it's cut the longest episode of Pragmatic ever. So thank you to the listeners and for making me do it. Woohoo! All right, we made it Vic and we're still alive. Woo! Yes. All right. Thanks, everyone. And thanks again, Vic. Thank you, John. Thank you, internet. Yes, thank you, internet. Oh, yeah, the internet rocks. [MUSIC PLAYING] [MUSIC PLAYING] (upbeat music) (upbeat music) ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ Do you have a cricket problem? Yes, I have cricket problems. That's what I thought. There's nothing I can do about it, man. No, I know there's nothing you can do about it. I was trying to determine if you were being invaded or if you were being attacked. if it was a fan somewhere. - No, it's a frigging, it's a bunch of crickets. And unless I do the Monty Burns push button and release the poison gas to kill the crickets, I ain't gonna. (laughing) - It's all good, John. - Oh no, I know, it's fine. It adds personality and character. (laughing) I think I'm nearly dead.