325: renderDEEZ128

00:00SPEAKER- Okay, I did some edge of the seat beep work there, Brad.

00:04SPEAKER- Yeah, what does that mean?

00:05SPEAKER- Well, I started the recording at like six seconds to go.

00:09SPEAKERI usually try to start it at about two seconds to go

00:11SPEAKERso that you don't have to, you know,

00:13SPEAKERthat I'm the short one instead of the long one.

00:15SPEAKER- Same.

00:16SPEAKER- And I realized I had to delete, I had to stop it,

00:19SPEAKERdelete the file that was there, the line that was there,

00:21SPEAKERand then restart it with like two seconds to go,

00:25SPEAKERand I got it with maybe half a second on the first beep.

00:29SPEAKER- The hidden beep etiquette that emerges

00:31SPEAKERwhen you podcast remotely all the time.

00:34SPEAKERWe're talking about the little,

00:35SPEAKERI think we refer to it internally as beep 30.

00:38SPEAKER- Yeah.

00:39SPEAKER- Which is the tool that was built for us by Thrist Heart.

00:42SPEAKERIs that how you say his username?

00:43SPEAKER- I think that's right, Thrist Heart, yeah.

00:44SPEAKER- On the Discord, which is the thing we use

00:47SPEAKERto sync our recordings together.

00:48SPEAKERAnyway, yes, I also, I always hit the record button

00:50SPEAKERlike right before the beep starts

00:52SPEAKERso that it's right at the beginning,

00:54SPEAKERand especially for if I'm not editing,

00:56SPEAKEReven when I am editing the podcast, it's useful,

00:58SPEAKERbut especially when I'm not right there at the front.

01:01SPEAKERYou don't have to look for it.

01:02SPEAKER- No, no, you don't wanna have to scroll a minute in

01:05SPEAKERand then find it, that's no good.

01:06SPEAKERBut-

01:07SPEAKER- That said, I know exactly what the waveform

01:09SPEAKERof those beeps look like.

01:09SPEAKERIt's like burned into my memory at this point.

01:11SPEAKER- Oh yeah, it's the most important waveform I ever see.

01:16SPEAKERI was gonna ask you.

01:18SPEAKERI don't actually remember what I was gonna ask you now

01:19SPEAKER'cause we started talking about the beeps

01:21SPEAKERand I was like there's a mosquito in my office

01:22SPEAKERthat I've gotta deal with.

01:24SPEAKER- The perspective cold open topics discussed

01:27SPEAKERright before we started this were

01:28SPEAKERthere's a mosquito in your office

01:30SPEAKERand you're down to two micro USB devices.

01:32SPEAKER- Oh, right, the micro USB devices.

01:34SPEAKERSo I have a perfectly good USB PD2 charging backpack.

01:43SPEAKER- I see where this is going.

01:45SPEAKERI already am experiencing the pain

01:48SPEAKERof whether this is worth it or not.

01:50SPEAKER- Yeah, it's great.

01:52SPEAKERIt will charge the entirety

01:54SPEAKERof anybody's phone in the house almost.

01:56SPEAKERI think Gina's big iPhone Max has a bigger battery than this

02:00SPEAKERbut my daughter's mini and my older iPhone Max

02:03SPEAKERboth will fully charge off of this thing.

02:07SPEAKERIt's a micro USB charger.

02:10SPEAKERAnd I have, there's literally on my charge station

02:12SPEAKERover here to the right, there's one yellow cord remaining

02:17SPEAKERthat's for the three micro USB charging devices.

02:21SPEAKERI have one thing that's, let's say, legal drug paraphernalia

02:28SPEAKERthat's micro USB, it's like a flower vape.

02:33SPEAKERIt's nothing weird, but yeah.

02:35SPEAKERAnd then I have, that I don't really use all that often

02:39SPEAKERso it's not really an issue.

02:40SPEAKERAnd then I have this battery backup

02:42SPEAKERand I have one other battery backup

02:44SPEAKERbut the other battery backup has both USB micro C

02:48SPEAKERand a micro charger so it's fine, it gets to live.

02:51SPEAKER- It can charge off either port?

02:53SPEAKER- It has two ports for charging and two ports for draining.

02:57SPEAKER- Oh, I thought typically the batteries

02:58SPEAKERonly have one port that would charge, that's cool.

03:01SPEAKER- Yeah, so it's from a weird time.

03:02SPEAKERIt's from like right as the changeover was starting

03:06SPEAKERbut before like iOS and switched over,

03:08SPEAKERiOS devices and switched over.

03:10SPEAKER- So what's the quandary?

03:11SPEAKERIs it whether to keep the micro only battery?

03:14SPEAKERI just saw the mosquito fly right across your face.

03:16SPEAKER- It's making me crazy, man. - You're fucked.

03:17SPEAKER- Yeah.

03:19SPEAKER- You're boned, I'm sorry to tell you.

03:20SPEAKER- Everything's collapsing here, man, it's bad.

03:24SPEAKER- What is it?

03:25SPEAKERIs the question whether to get rid

03:27SPEAKERof the micro only battery or?

03:29SPEAKER- Yeah, the question is, is it like I generally

03:33SPEAKERon stuff like batteries, I try to keep them

03:34SPEAKERuntil either they get puffy and stop working

03:36SPEAKERor they stop working, right?

03:37SPEAKER'Cause it's like, there's no downside to having a battery

03:40SPEAKERthat's like 80% capacity or whatever.

03:42SPEAKER- That is, well, I would say the same thing is true

03:44SPEAKERof a battery that is limited to micro USB.

03:47SPEAKERI'd say their batteries are too useful as a backup.

03:49SPEAKERI mean, I don't know about you,

03:51SPEAKERwe don't have that many power outages around here,

03:53SPEAKERalthough certainly more than we used to,

03:54SPEAKERbut it's just too useful to have extra batteries around.

03:58SPEAKERAnd even if you never use that one hands-on,

04:00SPEAKERlike even if that one is just a charge it once

04:03SPEAKERevery six months and leave it in the drawer for emergencies,

04:05SPEAKERI still feel like that's not a good reason

04:07SPEAKERto get rid of something that's otherwise quite useful.

04:09SPEAKER- Well, so the problem with this one

04:11SPEAKERis because it's the smallest kind of crappiest one

04:14SPEAKERthat's in service still, it's the one that when the kiddo

04:18SPEAKERis like, when I'm like,

04:18SPEAKER"Hey, did you charge your phone last night?"

04:20SPEAKERAnd she's like, "Oh yeah, I got it, I'm good.

04:21SPEAKER"I'm at 18% for being ready to go out of the house

04:23SPEAKER"for three hours."

04:25SPEAKERI'm like, "No, you need to take a battery backup

04:28SPEAKER"and plug it into your phone in your bag."

04:29SPEAKERAnd she's like, "Oh, I don't wanna do that."

04:31SPEAKER- Has she ever actually taken a battery with her?

04:33SPEAKER- Oh yeah, yeah, like when she goes to class or something

04:37SPEAKERand she has like 10% battery, it's always dead.

04:40SPEAKER"Can I borrow a battery?"

04:41SPEAKERAnd I don't wanna give her the $100 big chunky one

04:44SPEAKERthat'll do laptops and is good

04:46SPEAKERfor four or five phone charges.

04:48SPEAKERA, because it's heavy and big,

04:50SPEAKERand B, because it was kind of expensive

04:51SPEAKERand it's like, that's like our disaster preparedness one.

04:55SPEAKERThat's the one that I can take out.

04:56SPEAKERIf we have a bad problem, the power's out for a week again

04:59SPEAKERlike it was in 2020 when the fires happened,

05:02SPEAKERI can take that one out,

05:03SPEAKERI can charge it off the car battery

05:05SPEAKERand then bring it back in

05:06SPEAKERto top up everybody's phones and stuff.

05:08SPEAKERSo I feel bad getting rid of it.

05:12SPEAKERI feel like it's wasteful.

05:14SPEAKER- Sure, I kinda feel,

05:16SPEAKER"Well, you could give it away to somebody or maybe-"

05:18SPEAKER- I'm not gonna give somebody a USB,

05:20SPEAKERmicro USB battery charge. - Somebody might take it.

05:23SPEAKERAll I mean is if you don't wanna waste it,

05:24SPEAKERif you don't wanna literally junk it,

05:26SPEAKERyou could find somebody who would use it

05:27SPEAKERif you wanna feel like it's still being put to use.

05:30SPEAKER- Maybe I should use it for like a project.

05:31SPEAKERMaybe I should get something that's like,

05:33SPEAKERI saw one of those e-ink kind of ambient display screens

05:38SPEAKERthe other day and one of the options for that

05:40SPEAKERwas to hang a USB battery in it.

05:42SPEAKERAnd like, I bet that this battery

05:44SPEAKERwould run that thing for like four months.

05:45SPEAKER- Oh yeah, sure. - As little power as it draws.

05:47SPEAKER- My Kindle, I have not,

05:48SPEAKERI read my Kindle every night sometimes.

05:50SPEAKEROnly I usually only make it 10, 15 minutes

05:52SPEAKERbefore I fall asleep,

05:53SPEAKERbut I haven't charged my Kindle in like six months

05:55SPEAKERor something and I just noticed

05:56SPEAKERit was down to like 40% the other day,

05:58SPEAKERbut still that's like six months of Kindle usage, so yes.

06:01SPEAKERAnd that's with a backlight.

06:02SPEAKERSo I think like a non-backlight e-ink screen

06:05SPEAKERwould probably go forever.

06:08SPEAKER- Do you put your Kindle on airplane mode?

06:10SPEAKER- Sorry, yes, I should point out.

06:12SPEAKERIt's always an airplane mode unless I need to sync to it

06:14SPEAKER'cause the battery lasts so much longer.

06:16SPEAKER- Gotcha.

06:17SPEAKER- And also, this is a topic for another time,

06:19SPEAKERbut Amazon has gotten incredibly aggressive

06:21SPEAKERabout pushing auto updates to that thing.

06:24SPEAKER- I almost always just read on my phone or my iPad now.

06:28SPEAKER- Really?

06:29SPEAKER- But I also, like I tend to be,

06:32SPEAKERlike I find myself in places where I have like 10 minutes

06:34SPEAKERwhere I'm like waiting.

06:36SPEAKERThe classic example is, oh, I have to have the oil changed.

06:38SPEAKERI'm sitting there waiting for the oil change to be done,

06:41SPEAKERbut like that's not really a problem

06:42SPEAKERwith the electric car anymore.

06:44SPEAKERBut yeah, like if I'm in a doctor's office or something,

06:47SPEAKERI always sync 'cause I always wanna have my latest location

06:50SPEAKERso I can just pick it up on the phone.

06:51SPEAKER- Fair, yeah, I get it.

06:53SPEAKER- The magic for me is,

06:55SPEAKERalso I'm doing almost exclusively Libby these days.

06:57SPEAKERI don't think I've bought a book for Amazon in a year.

06:59SPEAKERActually, I know when the last time

07:01SPEAKERI bought a book for Amazon was.

07:02SPEAKERIt was when they did the changeover on--

07:05SPEAKER- Oh, the DRM stuff?

07:06SPEAKER- The DRM changeover, 'cause like--

07:08SPEAKER- I also, same, have not bought it.

07:10SPEAKERI successfully ran whatever that little JavaScript applet

07:14SPEAKERwas that pulled all my stuff off,

07:16SPEAKERwhich I have archived now.

07:17SPEAKERAnd I probably will never buy another book from them

07:20SPEAKERbecause of that.

07:21SPEAKER- Also, shout out to Libby for cookbooks.

07:24SPEAKERI didn't realize this.

07:26SPEAKERBut like San Francisco Public Library

07:28SPEAKERand San Mateo County Libraries both have really,

07:32SPEAKERreally robust e-cookbook collections.

07:36SPEAKERSo like when I was looking at the Tartine Bakery,

07:39SPEAKERthe fourth edition of it,

07:40SPEAKER'cause I have the second edition

07:42SPEAKERand the recipe I wanted isn't in that one.

07:43SPEAKERIt's only in the fourth edition.

07:45SPEAKERIt's like a $50 cookbook.

07:47SPEAKERAnd I was like, man, I'm gonna have to buy another,

07:48SPEAKERI don't wanna have another $50 cookbook on my shelves.

07:51SPEAKERAnd then I was like, what about Libby?

07:52SPEAKERI searched Libby, it's there.

07:54SPEAKERNobody had it checked out.

07:55SPEAKERThey had like eight copies of it.

07:56SPEAKERAnd I got it, I copied and pasted the recipe out

07:59SPEAKERand I put it in my paprika.

08:01SPEAKERAnd then I just returned the book immediately.

08:02SPEAKERIt was great.

08:03SPEAKER- Wait, hang on, is that a Libby interface

08:05SPEAKERto a local, like physical library system?

08:07SPEAKEROr is that just some like cloud Libby,

08:10SPEAKERcloud library that does not actually exist?

08:13SPEAKER- So you can do both, right?

08:15SPEAKERLike there's virtual libraries

08:17SPEAKERthat you can connect to with Libby.

08:18SPEAKERYou can also connect to your local system.

08:20SPEAKERLike when I check out an e-book

08:22SPEAKERfrom the San Francisco Public Library,

08:24SPEAKERI add it to my Kindle with Libby.

08:27SPEAKER- Hmm, okay. - Have you never done this?

08:28SPEAKERI assumed as a library aficionado,

08:30SPEAKERyou'd have been on this train a long time ago.

08:32SPEAKER- I've mostly been reading the same three.

08:35SPEAKERLike I said, I use books largely to fall asleep at night.

08:37SPEAKERI wish I had time to read more,

08:39SPEAKERbut I kind of have been reading the same three books

08:42SPEAKERoff and on.

08:43SPEAKERThey're like short story collections

08:44SPEAKERand one very long history book.

08:46SPEAKERI mean like a 700 page history book for a very long time.

08:49SPEAKERSo I just have not obtained any new books.

08:51SPEAKERYou're still looking around at that mosquito.

08:53SPEAKER- Fucking mosquito.

08:54SPEAKER- Anyway, I have not really used the SF library

08:59SPEAKERsince they started transitioning.

09:00SPEAKEROff of they were used to be on like overdrive.

09:03SPEAKER- Oh, Libby's dope.

09:04SPEAKER- Yeah, I should try it.

09:05SPEAKERI should get back into that.

09:07SPEAKER- It took like five minutes to connect my card

09:09SPEAKERto the system.

09:11SPEAKER- Just to close the loop on this whole cold open

09:12SPEAKER'cause we got to start the show.

09:14SPEAKERAnd this has been like five cold opens in one.

09:16SPEAKERThe real reason I fell off of messing with the Kindle

09:18SPEAKERis the last time I tried to jailbreak it.

09:20SPEAKER- Oh.

09:21SPEAKER- And to go back to the auto updates thing,

09:22SPEAKERmaybe we should do an update on a future episode about this.

09:25SPEAKERI washed out of the jailbreaking my Kindle process

09:28SPEAKERbecause it auto updated in the middle of it.

09:30SPEAKER- Oh my God.

09:31SPEAKER- To a version that could no longer be jailbroken.

09:33SPEAKER- Oh, that stinks.

09:34SPEAKER- And that kind of soured me on the whole experience so much

09:36SPEAKERthat I'm not using the Kindle as much

09:39SPEAKERfor the last little bit.

09:40SPEAKER- So you saying you tried to jailbreak your Kindle,

09:44SPEAKERbut you ended up putting your Kindle in jail?

09:45SPEAKER- Hmm, yeah, it's more like Kindle jail broke me.

09:51SPEAKERI don't know, let's podcast.

09:53SPEAKER(upbeat music)

10:00SPEAKER(upbeat music)

10:20SPEAKER- Welcome to Brad and Will Made a Tech Pod.

10:23SPEAKERI'm Will.

10:24SPEAKER- I'm Brad.

10:24SPEAKER- Hi Brad.

10:25SPEAKER- Hello.

10:26SPEAKER- Hi.

10:27SPEAKER- This week, we're doing something a little bit different.

10:30SPEAKERWe haven't done this.

10:31SPEAKERWe haven't talked about this in kind of a long time.

10:32SPEAKERWe also kind of talk about it a little bit all the time.

10:34SPEAKER- Yes, you're not wrong.

10:35SPEAKERBut this is, honestly, this episode is,

10:39SPEAKERthe genesis of this episode is by user request.

10:42SPEAKER- That's true.

10:43SPEAKER- We're gonna do kind of a,

10:44SPEAKERI guess this is our beginning of 2026

10:46SPEAKERkind of home lab update.

10:47SPEAKERI don't, I have to confess.

10:49SPEAKERI don't love the term home lab.

10:51SPEAKER- I mean-

10:52SPEAKER- I don't hate it.

10:53SPEAKERLike it communicates the concept well.

10:55SPEAKERActually, what I really don't like

10:56SPEAKERis having spent a lot of time

10:57SPEAKERon the true NAS forms in the past

10:59SPEAKERwhere people post gigantic SIGs

11:02SPEAKERof their like three massive servers

11:04SPEAKERwith 96 hard drives each.

11:06SPEAKER- Look, SIGs are always bad.

11:07SPEAKER- And like the term just evokes like,

11:12SPEAKERwhat's the concept I'm trying to say here?

11:14SPEAKERIt's not a form of a function thing,

11:17SPEAKERbut it's kind of analogous to that.

11:19SPEAKERLike a-

11:21SPEAKER- Well-

11:22SPEAKER- What is the term for like amassing way more capability

11:25SPEAKERthan you actually need?

11:26SPEAKERLike way more?

11:26SPEAKER- Overkill?

11:27SPEAKER- Sure.

11:28SPEAKER- You know what I mean?

11:29SPEAKER- Sure.

11:30SPEAKER- There's an arms race in the,

11:33SPEAKERlike if you go to r/homelab or one of those places

11:35SPEAKERwhere it's like somebody posts their 36 inch tall rack,

11:42SPEAKERyou know, a 24U rack,

11:43SPEAKERand then somebody else is like,

11:45SPEAKERhey, check this out.

11:46SPEAKERMy work was thrown this out and I got a 48 inch,

11:49SPEAKER48U, 12 foot tall.

11:52SPEAKERIt's gonna be 1500 pounds

11:54SPEAKERwhen I put all the computers in it rack.

11:55SPEAKER- Yeah, my one petabyte storage server.

11:57SPEAKERYou know what it is?

11:58SPEAKERIt's hot riding.

11:59SPEAKERThat's all it is.

12:00SPEAKERThat's all it is.

12:01SPEAKERIt is just tricking something out

12:02SPEAKERway beyond any practicality or necessity.

12:06SPEAKER- But I mean, I think that there's another,

12:08SPEAKERso there's multiple aspects to the whole home lab thing.

12:11SPEAKER'Cause the other thing is people use them as learning places.

12:14SPEAKERAnd for example, back in the old days at Maximum PC,

12:19SPEAKERwe never, our business people, our ad sales people,

12:22SPEAKERwere always like,

12:23SPEAKERhey, could you do the thing that PC World does

12:25SPEAKERand have like a big info world type,

12:28SPEAKERcan you do some server testing and stuff like that?

12:30SPEAKERAnd we're like, no, we're not set up to do that.

12:31SPEAKERIt's too expensive.

12:33SPEAKERLike we don't have the capacity to build the 300 machines

12:36SPEAKERwe need to test a web server effectively, right?

12:40SPEAKEROr whatever it was.

12:42SPEAKERAnd the home lab community has kind of used,

12:48SPEAKERI don't know, they're not special purpose,

12:50SPEAKERbut like used computers and stuff like that

12:54SPEAKERand come up with some best practices

12:56SPEAKERthat let you do that kind of stuff

12:59SPEAKERand build that kind of expertise and build like,

13:02SPEAKERhey, I'm building a big giant web infrastructure

13:05SPEAKERor server infrastructure

13:07SPEAKERso that you have a place to practice that and test that

13:09SPEAKERand fool around with that as a personal person,

13:11SPEAKERwhich is cool. - Totally.

13:12SPEAKERI mean, A, okay, A, to be clear,

13:14SPEAKERpeople can do whatever they want with their money and time.

13:17SPEAKERB, if they're actually making practical use

13:20SPEAKERof the thing they have built even better.

13:22SPEAKERBut I just personally don't want to feel

13:24SPEAKERlike I am building all this stuff out

13:27SPEAKERjust to sit there and admire it.

13:28SPEAKERAnd like, I kind of try to make sure

13:30SPEAKERI'm actually making use of what I'm doing.

13:32SPEAKERLike, for example, I kind of overbought hard drives.

13:35SPEAKER- Yeah. - Year before last

13:36SPEAKERat Black Friday, I expanded my storage pool.

13:39SPEAKER- Yeah. - To the point that I like,

13:41SPEAKERit's still sitting there half empty

13:43SPEAKERbecause at some point I finally realized,

13:45SPEAKERyou know, I may have bought too many hard drives

13:46SPEAKERat this point.

13:47SPEAKER- So to me, it's like the people who are the hot rodders,

13:51SPEAKERthe people who just want to have like the biggest,

13:53SPEAKERbaddest machine set up, right?

13:55SPEAKERAnd then there's the piracy aficionados.

13:57SPEAKERThey're running a whole stack of ours.

13:59SPEAKER- See, so those people with petabyte storage volumes

14:01SPEAKERare actually using it for something.

14:03SPEAKER- Yeah, like-- - The ethics

14:05SPEAKERof that something aside.

14:06SPEAKER- Yeah, it's like Steve from North Carolina

14:10SPEAKERwants to roll his own Netflix.

14:12SPEAKERGot it, I understand that.

14:13SPEAKERIt's not for me, but I get it.

14:15SPEAKERAnd then there's the people that want to learn.

14:17SPEAKERThere's like the people that are like,

14:18SPEAKERare you building stuff so they can get certifications

14:21SPEAKERor advance in their job or whatever, which is great.

14:22SPEAKER- Kind of, I guess that's kind of where I fall these days.

14:25SPEAKERI mean, I'm not trying to get certifications,

14:27SPEAKERbut like I was going through some networking stuff

14:30SPEAKERin the network channel the other day

14:31SPEAKERand people started talking about having done that stuff

14:33SPEAKERfor their CCNA.

14:35SPEAKERIt was a bunch of IPv6 stuff that I was poking around at.

14:38SPEAKERAnd it was just like, okay, I guess that's kind of

14:39SPEAKERwhat I use this stuff for is I just like learning.

14:42SPEAKER- Well, and then there's also people who want to kind of

14:44SPEAKERget off of the public, like the corporate clouds, right?

14:47SPEAKERSo there's people who want to divest of Google.

14:50SPEAKERI think I fall in that category a little bit

14:52SPEAKERor at least have an alternative too,

14:55SPEAKERin case you decide you want to pull the plug entirely.

14:57SPEAKERAnd so, okay.

15:00SPEAKERSo there's motivations, that's what people are doing.

15:02SPEAKER- Yeah, so real quick, this all started from James

15:05SPEAKERon the Discord adding me a question, which was,

15:09SPEAKERhey, can you run through your Linux NAS,

15:11SPEAKERlike your Linux server software stack?

15:13SPEAKERBecause I run TrueNAS, I'm speaking as James here.

15:17SPEAKERThis is James saying, I run TrueNAS,

15:18SPEAKERbut I'm like kind of over it

15:20SPEAKERor I'm kind of thinking about moving to something else.

15:22SPEAKERAnd I'm enticed by this idea of just running

15:24SPEAKERa bare operating system and doing everything yourself,

15:26SPEAKERbut I'm also kind of afraid or intimidated by it.

15:29SPEAKERAre you gonna get it?

15:30SPEAKER- No. - He's leaning back.

15:32SPEAKER- You gotta stop asking me about it, it's an audio podcast.

15:34SPEAKER- I know, but this is like-

15:36SPEAKER- I know it's high drama. - This is action cast.

15:38SPEAKERYes, high drama is the target.

15:39SPEAKER- It loves the lights.

15:40SPEAKERI'm gonna go and tell you it loves my lights.

15:42SPEAKER- It's a flying bug. - Yeah.

15:43SPEAKER- Of course it does.

15:44SPEAKER- I need a bug assault.

15:46SPEAKER- It's hard not to interject

15:47SPEAKERwhen you assume the pose,

15:49SPEAKERyour hands came up in the clapping formation

15:51SPEAKERas if you were about to get it

15:52SPEAKERand I just couldn't help myself.

15:54SPEAKERAll right, anyway, James was basically asking,

15:57SPEAKERhey, can you step through like the software stack

15:59SPEAKERand how you've configured everything on your Linux box

16:02SPEAKERto be a NAS slash server

16:04SPEAKERwithout the aid of an appliance style operating system?

16:07SPEAKERAnd that was kind of the genesis of this episode,

16:09SPEAKERbut we're gonna kind of expand it out a little bit

16:12SPEAKERand just sort of do a quick home lab overview of everything.

16:15SPEAKER- Yeah, well, I thought it was interesting to kind of

16:18SPEAKERdo a reset 'cause like long time listeners will know

16:21SPEAKERthat in a lot of ways, this podcast started

16:24SPEAKERwith us talking about home labs.

16:26SPEAKERWe started talking about TrueNAS, FreeNAS back then.

16:28SPEAKER- Sure, yes, 'cause this podcast started,

16:31SPEAKERI built my TrueNAS machine in April of 2018

16:35SPEAKERand we started this in September of '19.

16:38SPEAKER- Yeah, the origin for a lot of this

16:40SPEAKERwas you and I were spending a lot of time on Discord

16:41SPEAKERtalking to each other about server stuff

16:43SPEAKERand finally we were like, I was like,

16:44SPEAKERhey, do you think CBS would let you do a podcast?

16:47SPEAKERYou were like, I don't know, let me ask.

16:49SPEAKERAnd then you asked and then it took them a year to decide

16:52SPEAKERand then we did a podcast.

16:53SPEAKER- Yeah, yeah, and you and Vinny were the people I knew

16:57SPEAKERrunning FreeNAS, like that's the whole reason

16:59SPEAKERI got into that in the first place.

17:01SPEAKER- I'd been running FreeNAS at that point

17:03SPEAKERfor like five or seven years, I don't know,

17:06SPEAKERsince I went from Windows home server to FreeNAS

17:08SPEAKERas I recall, so.

17:10SPEAKER- Which became TrueNAS to be clear.

17:11SPEAKER- Yeah, the classic path.

17:12SPEAKERBut anyway, the thing that's happened though

17:15SPEAKERis we've done, like our stuff has evolved.

17:18SPEAKERYou know, when we started this I was running FreeNAS

17:21SPEAKERon a Broadwell E, like my old gaming machine

17:25SPEAKERwhich was a Broadwell E eight core machine

17:28SPEAKERthat was like using 500 watts of power at idle

17:31SPEAKERwith no hard drive spun up or something ridiculous.

17:33SPEAKERAnd our setups have evolved, our hardware's evolved,

17:38SPEAKERthe services we're running have evolved

17:39SPEAKERand we haven't really talked about it as a whole

17:41SPEAKERin a long time.

17:43SPEAKERSo rather than make people listen to 300 episodes

17:47SPEAKERand piece together the gradual evolution

17:49SPEAKERof what we're doing, we thought it was useful

17:51SPEAKERto like run down the stacks and kind of talk about it.

17:55SPEAKER- Yes, should we start with my server?

17:57SPEAKER- Yeah, we can start with your server.

17:58SPEAKER- Since that was the question.

17:59SPEAKER- You wanna go server, you wanna go one thing,

18:03SPEAKERyou wanna ping pong back and forth?

18:04SPEAKER- I don't actually have a ton of other stuff

18:06SPEAKERso I'm mostly just gonna focus on my server.

18:08SPEAKERI will just like briefly run down the other boxes

18:10SPEAKERdoing stuff in my house at the end of this

18:12SPEAKERbut I've talked about those enough recently

18:14SPEAKERthat I feel like just touching on that should be fine.

18:17SPEAKERBut there's a lot of meat to administrating

18:20SPEAKERthis big Linux box.

18:23SPEAKER- The interesting thing is we've both taken

18:25SPEAKERdifferent approaches 'cause like you've gone

18:27SPEAKERwith one monolithic server, I have a bunch of stuff

18:29SPEAKERspread out over a bunch of low power devices.

18:31SPEAKER- Right, and you're a bit more appliance.

18:33SPEAKERWell, I think your kind of VM host

18:35SPEAKERis just a bare operating system, we'll get to that.

18:39SPEAKER- Okay, all right, my current server

18:41SPEAKERis an Alder Lake Core i5 12600K.

18:44SPEAKERI'll just briefly run down the specs here.

18:46SPEAKERI've got a very good deal on that i5, like very good

18:50SPEAKERor otherwise I probably would have gone

18:51SPEAKERfor like an i3, something lower power.

18:54SPEAKER- Well-- - Go ahead.

18:57SPEAKER- I was just saying having gone

18:58SPEAKERwith the really low power solution,

19:00SPEAKERit's nice to have extra compute.

19:02SPEAKER- Yeah, occasionally compile software on there

19:05SPEAKERand stuff like that and it's got a quick sync.

19:07SPEAKERAnyway, okay, so it's an i5 12600K.

19:10SPEAKERThere's 64 gigabytes of ECC DDR5 in there.

19:14SPEAKERThat's ECC is error correcting RAM.

19:16SPEAKERI had to get an ASUS workstation board

19:18SPEAKERwith a W680 chip set to get--

19:20SPEAKER- But that was expensive.

19:22SPEAKER- Actually, it was about 330,

19:23SPEAKERwhich is very cheap in that world.

19:26SPEAKERI wouldn't have gone for the ECC until ASUS rolled out

19:29SPEAKERthat lower end workstation board is the reason

19:33SPEAKERI went for it and the RAM was barely more expensive

19:35SPEAKERthan regular RAM at the time.

19:37SPEAKERAlso, this was three years ago when RAM was a commodity,

19:41SPEAKERlike a dirt cheap.

19:42SPEAKER- Yeah, yeah, those were good days.

19:44SPEAKER- Yeah, a ton of hard drives in there.

19:45SPEAKERI've talked about it before.

19:46SPEAKERThere's eight hard drives mirrored in the main pool.

19:49SPEAKERSo there's only four drives worth of space,

19:52SPEAKERwhich comes out to 65 terabytes.

19:54SPEAKER- Good god damn. - Total, well, yes, yes.

19:56SPEAKERI have been amassing Black Friday hard drive deals

19:58SPEAKERfor a couple of years.

19:59SPEAKERAnd like I said, it's too much now

20:00SPEAKERbecause I don't actually need that much, it turns out.

20:04SPEAKERBut anyway, there's also a couple NVMe drives in there

20:07SPEAKERthat I use as scratch space.

20:09SPEAKER- What do you mean scratch space?

20:10SPEAKER- Like, kind of anything and everything.

20:14SPEAKERLike I record, like when I record videos in OBS of streams

20:19SPEAKERor like when I record the ramble cast,

20:20SPEAKERwhich I do a video of every week,

20:22SPEAKERlike it records to that rather than the spinning drives

20:25SPEAKERbecause no access time and I don't have to worry

20:27SPEAKERabout like anything seizing up in the IO chain.

20:32SPEAKER- And you're directly connected from your desktop PC

20:35SPEAKERto the NAS with like a high speed connection?

20:36SPEAKER- I'm over this 40 gigabit connection to the NAS.

20:39SPEAKERSo it's pretty robust for sending large amounts of data.

20:42SPEAKERBut frankly, even that amount of data

20:44SPEAKERcould go over a one gigabit link pretty much, just fine.

20:48SPEAKER- You say that, but like I find that OBS

20:51SPEAKERis really sensitive to network latency.

20:53SPEAKERSo that like, if the machine, for example,

20:56SPEAKERwhen I tried to run all of my OBS setup stuff

20:59SPEAKERoff of the NAS, booting up OBS,

21:02SPEAKERsometimes it would just be weird

21:04SPEAKERbecause it would like, the drives would be spinning up

21:07SPEAKERor it would take a second to find a file or something

21:09SPEAKERand it would completely, it didn't like that

21:11SPEAKERin ways that caused unpredictable bugs, so.

21:14SPEAKER- Yeah, I've never had an issue with it,

21:15SPEAKERbut like I also use that scratch space.

21:17SPEAKERSo those are two terabyte NVMe drives

21:19SPEAKERthat are currently are mirrored

21:21SPEAKERbecause I basically decided everything in this machine

21:23SPEAKERshould be mirrored since I use it for work

21:25SPEAKERand other kind of critical stuff a lot.

21:27SPEAKERAnd I just don't ever want to drive failure

21:28SPEAKERto take anything down because there's not a lot

21:31SPEAKERon that mirrored volume, NVMe volume

21:34SPEAKERthat is super duper important.

21:35SPEAKERAll those recordings are pretty important,

21:37SPEAKERbut I have considered striping that volume,

21:40SPEAKERwhich would make it four terabytes

21:41SPEAKER'cause striping is no redundancy versus,

21:44SPEAKERanyway, two terabytes, still plenty of scratch space.

21:47SPEAKER- What if you added a third NVMe and then did parodies

21:50SPEAKERso you could have both? - I could do that.

21:52SPEAKER- Yeah. - I could do that.

21:53SPEAKER- Think about it. - I don't know

21:54SPEAKERthat you mentioned it, but I'll also use it for things

21:56SPEAKERlike if I downloaded a ton of PlayStation ISOs

22:01SPEAKERfor Mr. Stream a couple of years ago, like a ton of them.

22:05SPEAKERAnd it's nice to have an extra high-speed drive in there

22:08SPEAKERto like, for example, like if I downloaded,

22:11SPEAKERlet's say like a couple terabytes of disc ISOs,

22:16SPEAKERreading those and writing them to,

22:18SPEAKER'cause they're all compressed,

22:19SPEAKERthey're all like seven zipped or whatever,

22:20SPEAKERreading and writing those to decompress them

22:22SPEAKERon the same drives is pretty slow.

22:24SPEAKERBeing able to deal, like have an intermediary drive

22:27SPEAKERin between. - Go from one drive

22:27SPEAKERto another, yeah, yeah, yeah. - It's way, way faster

22:29SPEAKERto have that extra drive space in there, so anyway.

22:33SPEAKER- That kind of stuff I all do on my desktop because of that.

22:36SPEAKERLike it's too slow going across the gigabit to the network.

22:40SPEAKER- Lastly, two SATA, two cheapo $20 SATA SSDs in there

22:44SPEAKERthat are mirrored as the operating system volume.

22:47SPEAKER- Okay, so that's your root and then your hard drives.

22:50SPEAKERAre your hard drives, is the big stack of hard drives

22:52SPEAKERin a ZFS position? - Yes, yes.

22:54SPEAKERYeah, it's all a big ZFS pool.

22:57SPEAKER- Wow, okay. - Okay, so why did I stop

23:00SPEAKERusing TrueNAS? - Yeah, hey Brad,

23:02SPEAKERwhy did you stop using TrueNAS?

23:03SPEAKER- Well, I'm glad you asked 'cause this is kind of what James

23:06SPEAKERwas getting at, like the simplest way I can explain it

23:10SPEAKERis that I just kind of got tired of chafing up against

23:13SPEAKERthe limitations that an appliance type operating system

23:16SPEAKERputs on you and like those, to be clear,

23:18SPEAKERthose limitations are there for a reason.

23:19SPEAKERIn fact, they are the whole reason

23:21SPEAKERthat an appliance style thing works in the first place.

23:23SPEAKER- The strength of the appliance style OS, honestly.

23:25SPEAKERSo when you say appliance, you mean stuff like Proxmox

23:30SPEAKERor Unraid or TrueNAS or it's like something that is,

23:33SPEAKERlike you install it and then you manage it probably

23:35SPEAKERthrough a web interface or an application

23:37SPEAKERrather than like a Linux console.

23:39SPEAKER- Right, and crucially, it is a layer,

23:43SPEAKERthose things are just layers on top of the same technologies

23:45SPEAKERwe're about to talk about, the same software stack.

23:47SPEAKERIt's just that, like you said,

23:48SPEAKERthey give you a nicer interface,

23:51SPEAKERthey unify everything a little more

23:52SPEAKERand crucially, like I said,

23:53SPEAKERthey put up a bunch of guard rails

23:55SPEAKERthat you're not supposed to go outside of

23:57SPEAKERbecause they need a known state for the system

24:01SPEAKERin order for everything to continue working together.

24:03SPEAKER- Well, and so you can do things like update

24:06SPEAKERand stuff like that.

24:07SPEAKER- That's exactly it.

24:07SPEAKERSo like, for example, like TrueNAS,

24:09SPEAKERyou're not supposed to go tinkering around

24:11SPEAKERwith the underlying operating system

24:12SPEAKERbecause even in the best case,

24:14SPEAKERlike a TrueNAS like system update

24:16SPEAKERwill just wipe out all the changes you made.

24:18SPEAKERIn the worst case, you will break something irreparably

24:20SPEAKERand have to start over.

24:21SPEAKER- Yeah, like this was, I think,

24:23SPEAKERone of the things I actually really liked about TrueNAS.

24:26SPEAKER- Yes, it is 100% just a philosophical approach.

24:28SPEAKERDo you want to really, do you want to be a mechanic

24:30SPEAKERand like being on your back under your car,

24:33SPEAKERtinkering with carburetor and changing the oil yourself,

24:35SPEAKERor do you want your car to just go,

24:37SPEAKERis kind of, I guess, the best metaphor?

24:39SPEAKER- So I dropped TrueNAS,

24:43SPEAKERjust while we're talking about it,

24:44SPEAKERI dropped TrueNAS at around the same time,

24:47SPEAKERbut for completely different reasons.

24:50SPEAKERMy hardware started getting janky

24:53SPEAKERand my Broadwell machine started getting wobbly

24:57SPEAKERand when I was looking at building a new machine for that,

25:01SPEAKERI was like, oh, let me look at the power requirements

25:03SPEAKERfor a desktop PC and what I'm paying for electricity now

25:07SPEAKERand I was like, oh, I don't want to do that anymore.

25:09SPEAKERWhat's the less power consuming option here?

25:14SPEAKERAnd I went out and bought a Synology NAS, honestly,

25:17SPEAKER'cause I was like, this has a decent four core Celeron

25:22SPEAKERor something in it, it has quick sync,

25:23SPEAKERit has all the things that I thought should let it run

25:26SPEAKERmost of the applications that I was running

25:28SPEAKERon that Broadwell machine without a whole lot of,

25:32SPEAKERmaybe things like compiling updates and stuff

25:34SPEAKERwould be a little bit slower,

25:36SPEAKERbut the day-to-day operations would be about the same.

25:38SPEAKERNow, in practice, not true.

25:40SPEAKERThe Celeron, the NAS that I bought

25:42SPEAKERwas a little too underpowered for that.

25:45SPEAKERBut in terms of like power consumption for serving files,

25:51SPEAKERit's way, way, way, way, way less money every month

25:54SPEAKERfor the, like to the tune of like 20 bucks.

25:56SPEAKER- Yeah, like I would guess the Synology

25:58SPEAKERis not using that much more power

26:00SPEAKERthan it takes to spin the hard drives.

26:01SPEAKERObviously there is overhead to run the electronics

26:03SPEAKERand the CPU and stuff in there,

26:04SPEAKERbut like contrast with my Wolf 600K that's probably using.

26:10SPEAKER- It's probably a hundred watts at idle, I guess.

26:13SPEAKER- Or I'm thinking even without the drives,

26:14SPEAKERwithout the drives, yeah.

26:15SPEAKER- Yeah, it's not insubstantial.

26:18SPEAKERThere's a reason I talk about wishing I had solar a lot.

26:20SPEAKER- So the Celeron, it's a J4125,

26:25SPEAKERwhich is a mobile Celeron in this one.

26:28SPEAKERGemini Lake refresh, which I think that's after Alder,

26:31SPEAKERbut I can't remember, is a 10 watt TDP.

26:34SPEAKERSo it's like, yeah.

26:37SPEAKER- Yeah, versus mine.

26:38SPEAKERI don't know what it is at idle,

26:39SPEAKERbut I mean, I think that 12600K is like 150 watt TDP.

26:43SPEAKERI mean, it's not running at that all the time, of course,

26:45SPEAKERbut okay, should we get into the software stack?

26:48SPEAKERThis is the thing James was actually asking about.

26:50SPEAKEROkay, so I basically just got tired of not being able

26:53SPEAKERto like do whatever I wanted on a machine that I owned.

26:57SPEAKERAnd again, if you just want a thing to work,

27:00SPEAKERthe appliance stuff makes perfect sense,

27:01SPEAKERbut I was constantly going like,

27:03SPEAKERman, I wish I could install this and use it,

27:04SPEAKERbut it's not part of what they have integrated

27:07SPEAKERinto this thing.

27:07SPEAKEROr like, oh, I wish this scheduling thing worked better,

27:10SPEAKERet cetera, et cetera.

27:11SPEAKERLike there's a new piece of software that I wanna try.

27:14SPEAKERIt's just, you couldn't do that with TrueNest.

27:16SPEAKER- You know what it really was,

27:17SPEAKERwas they're scheduling for like backups and scrubs,

27:20SPEAKERwhich scrubbing in ZFS terms is like checking

27:23SPEAKERthe data integrity on the drives.

27:25SPEAKERLike their scheduling was not as robust as I wanted.

27:28SPEAKER- For me, the big challenge with the BSD,

27:31SPEAKER'cause the thing about the time that we switched over,

27:33SPEAKERthe time I switched over at least,

27:34SPEAKERI was running TrueNest, the BSD version.

27:37SPEAKER- Which they are like killing us.

27:39SPEAKERWe don't need to get into this tangent,

27:40SPEAKERbut they are basically killing off

27:42SPEAKERthe FreeBSD product at this point.

27:43SPEAKER- I mean, it makes sense.

27:44SPEAKER- Yeah, yeah, yeah.

27:46SPEAKER- Because the problem with it was stuff would mostly work

27:50SPEAKERexcept for when it didn't, right?

27:52SPEAKERLike you could probably cross-compile a piece

27:55SPEAKERof Linux software for it, but it would be janky or weird,

27:58SPEAKERor you'd have to like jump through a bunch of hoops.

28:00SPEAKER- Or you could run a Linux VM,

28:03SPEAKERwhich I did for a little while, but that's extra overhead.

28:05SPEAKER- Then that's extra overhead, yeah.

28:06SPEAKER- Which was not great either.

28:07SPEAKERAlthough to be clear, the scheduling thing I was talking

28:10SPEAKERabout is TrueNest's UI middleware.

28:12SPEAKER- The built-in one, yeah.

28:13SPEAKER- I was just like, I wanted to be at like,

28:16SPEAKERI wanted to only scrub on the first Sunday

28:18SPEAKERof every second month or something like that,

28:20SPEAKERand they wouldn't let you do stuff like that.

28:22SPEAKERAnd then I started thinking like,

28:22SPEAKERyou know, that wouldn't be that hard

28:23SPEAKERto just script that yourself

28:25SPEAKERif I had the liberty to do that.

28:27SPEAKERI've just used the word liberty.

28:28SPEAKERThis is getting too ideal.

28:30SPEAKER- It's all about freedom, Brad.

28:32SPEAKER- This is getting too ideological, I'm afraid.

28:33SPEAKERBut anyway, okay.

28:35SPEAKER- Okay.

28:36SPEAKER- So I decided to go bare operating system.

28:38SPEAKERI actually was going to move to FreeBSD first

28:40SPEAKERbecause that's what I was used to,

28:41SPEAKERbut it had basically zero support

28:44SPEAKERfor the Alder Lake at the time, three years ago.

28:46SPEAKER- Of course.

28:47SPEAKER- And looking it up recently,

28:48SPEAKERthey are only now starting to get like full

28:51SPEAKERheterogeneous core scheduling support integrated.

28:55SPEAKERLike FreeBSD is cool,

28:56SPEAKERbut it just does not have the development support

28:58SPEAKERto even come close to catching up with Linux

29:00SPEAKERin terms of hardware.

29:02SPEAKER- Yeah.

29:02SPEAKER- So that's what drove me to Linux.

29:03SPEAKERIt was like, all right, it's time.

29:05SPEAKERThis is what the world runs.

29:06SPEAKERThis is what all the software is made for.

29:07SPEAKERIt's time.

29:08SPEAKER- I mean, look, there's a real argument in this space

29:11SPEAKERfor doing the thing that's,

29:12SPEAKERlike if you want to learn a bunch of stuff,

29:14SPEAKERthen you should do the thing you want to learn, right?

29:16SPEAKER- Yeah.

29:17SPEAKER- But if you want to have something

29:17SPEAKERthat's a usable functional computer,

29:19SPEAKERthere's a real argument for doing the thing

29:21SPEAKERthat everybody does.

29:23SPEAKER- Yep, yep, you're not wrong.

29:24SPEAKERYou're not wrong, okay.

29:25SPEAKER- Stick to defaults.

29:26SPEAKER- Yes.

29:27SPEAKER- What are you doing for file system?

29:28SPEAKER- Well, it's still ZFS.

29:30SPEAKERSo I'm running Debian stable.

29:34SPEAKER- Okay.

29:35SPEAKER- Debian 13 Trixie is the one that just came out last fall.

29:37SPEAKERSo I'm on that now.

29:39SPEAKERI picked Debian because it's stable.

29:41SPEAKERIt says it's stable right there in the name,

29:42SPEAKERbut like it is known to be the most like boring,

29:45SPEAKERslow, doesn't update often,

29:47SPEAKERand everything is supposed to work together properly

29:49SPEAKERand tested well distro out there pretty much.

29:52SPEAKER- I think in this case, that's a strength, not a weakness.

29:55SPEAKER- So I am very much in the camp of wanting a server

29:58SPEAKERto be boring and stable and reliable.

30:01SPEAKERThat said, I've been through,

30:03SPEAKERthis is my third or fourth, third Debian, I guess,

30:06SPEAKERsince I made this move and like every version of Debian,

30:08SPEAKERthere's at least one major package

30:11SPEAKERthat they don't have packaged or is way out of date

30:13SPEAKERthat I really wish I had access to and don't.

30:16SPEAKERSo there are trade-offs with everything, of course.

30:18SPEAKERI could move to a more aggressive distro.

30:20SPEAKERI just have not.

30:23SPEAKERYeah, so I am still using ZFS

30:24SPEAKER'cause that's what I was used to from TrueNAS.

30:26SPEAKERZFS is the big enterprise, you know,

30:29SPEAKERcheck summing file system that prioritizes data integrity

30:33SPEAKERand stuff like that.

30:34SPEAKERI don't need to get too deep into that for the moment.

30:37SPEAKERBut so you can install ZFS straight from Debian repos.

30:39SPEAKERThat's not a problem.

30:40SPEAKERYou do have to like agree to, hey, this license is not GPL.

30:43SPEAKERAre you okay with that?

30:44SPEAKERBut that's fine.

30:46SPEAKER- Why did you, did you say that you installed ZFS

30:50SPEAKERon the root?

30:50SPEAKER- So yes, I somewhat controversially

30:52SPEAKERin the Linux channel on our Discord,

30:54SPEAKERI am even running ZFS as the file system for the root volume.

30:58SPEAKER- I don't know about that, Brad.

30:59SPEAKER- That requires a lot of jumping through hoops.

31:01SPEAKERProbably- - Yeah,

31:02SPEAKERthat seems like a pain in the butt.

31:03SPEAKER- That's probably outside the scope of this episode, frankly.

31:06SPEAKER- Yeah, I mean, I get, so the benefit of ZFS is it's similar.

31:10SPEAKERIt's a journaling file system.

31:11SPEAKERYou can roll stuff back.

31:12SPEAKERYou can, like, if you-

31:15SPEAKER- Or it's copy on write is actually the term

31:18SPEAKERfor what it does.

31:19SPEAKERSo it was actually maybe to put it in terms that say

31:22SPEAKERlike Dual Boot Diaries listeners might understand,

31:25SPEAKERButterFS was kind of made in the spirits of ZFS.

31:30SPEAKERZFS came out of Sun like 20 years ago

31:32SPEAKERbefore Sun got gobbled up by Oracle.

31:34SPEAKERButterFS is very much doing the same type of stuff

31:37SPEAKERthat ZFS started-

31:38SPEAKER- In terms of snapshots and volumes and stuff like that.

31:44SPEAKER- Logical volume management, the whole thing.

31:46SPEAKERBCacheFS, which we've talked about,

31:47SPEAKERis also very much trying to implement

31:50SPEAKERa similar feature set to ZFS.

31:52SPEAKER- It kind of abstracts out a lot of file system stuff

31:55SPEAKERso that you can do things like roll back

31:57SPEAKERto previous versions of files

31:59SPEAKERor roll back to a previous state of the entire file system,

32:02SPEAKERstuff like that for people who don't know about this.

32:04SPEAKER- It's also, and again, this is again,

32:05SPEAKERa little out of scope here.

32:06SPEAKERIt is an entirely vertically integrated file system

32:08SPEAKERand volume management tool.

32:09SPEAKER- I don't know what that means.

32:10SPEAKER- Well, so as opposed to traditional file systems

32:16SPEAKERwhere like you need a raid management layer

32:18SPEAKERunderneath the file system layer.

32:20SPEAKER- Oh, right.

32:21SPEAKER- These copy on write file systems do everything.

32:23SPEAKERThey handle all the partitioning, all of the volume,

32:25SPEAKERthe physical volume management

32:26SPEAKERand the file system stuff on top of that.

32:28SPEAKER- And the distribution of data across the physical disks.

32:30SPEAKER- Right, yes, so.

32:31SPEAKER- So you can do things like just have a bunch

32:33SPEAKERof different sized disks jammed into your ZFS pool

32:36SPEAKERand they work, theoretically?

32:37SPEAKER- Yeah, yes, there's management there.

32:39SPEAKERI mean, so this is one of the things James was asking about

32:41SPEAKERbecause this is one of the things

32:42SPEAKERthat TrueNAS both abstracts for you

32:44SPEAKERand also makes relatively friendly with the web UI.

32:48SPEAKER- In fairness, Unraid and Proxmox,

32:50SPEAKERUnraid also does this, Proxmox I think doesn't.

32:52SPEAKER- Unraid got ZFS fairly recently

32:54SPEAKERin the last couple of years, I think.

32:55SPEAKERProxmox, I believe does have ZFS support of some kind.

32:58SPEAKERI'm not an expert on that stuff,

32:59SPEAKERbut yes, to address the point,

33:02SPEAKERI have had to learn a lot

33:03SPEAKERabout the ZFS command line tool chain.

33:06SPEAKERI mean, a lot, like perhaps more than I,

33:09SPEAKERI shouldn't say more than I ever wanted to know

33:11SPEAKERbecause I love knowing this stuff.

33:12SPEAKERLike it's kind of the whole reason I'm here.

33:14SPEAKERBut I, yes, you will spend a lot of time understanding,

33:18SPEAKERunderstanding how this stuff, I mean,

33:20SPEAKERit's cool if you enjoy this stuff

33:22SPEAKERbecause I think that the tooling around ZFS

33:26SPEAKERthat's provided at the command line

33:27SPEAKERis incredibly elegant and does some really cool stuff.

33:30SPEAKERBut it is a lot of learning.

33:31SPEAKER- See, I looked at this,

33:33SPEAKERI looked at the work you were doing on this

33:34SPEAKERand I was like, I'm gonna go buy a thing

33:36SPEAKERthat I can configure with a web interface.

33:37SPEAKER- Yeah, totally valid.

33:38SPEAKER- Yeah.

33:40SPEAKER- I mean, you have to be down for this as a hobby

33:42SPEAKERif you're gonna go this route.

33:44SPEAKERRunning that root volume on ZFS

33:46SPEAKERrequired me to basically do a manual install of Debian.

33:50SPEAKERIt's very similar to the Arch install if you've ever done that.

33:52SPEAKER- It's all command line based.

33:53SPEAKERIt's like the old school Debian install, right?

33:55SPEAKER- Yeah, you basically have to build up

33:56SPEAKERthe kind of ZFS volumes manually from like a live CD

34:01SPEAKERand then manually install everything.

34:03SPEAKER- Having done a fair number of Linux installs

34:05SPEAKERin the last year, six months,

34:07SPEAKERlet me go ahead and tell you,

34:09SPEAKERI don't like the partition manager

34:11SPEAKERin any of the graphical Linux installs

34:13SPEAKER'cause they don't give you granular enough control

34:15SPEAKERover anything and you always have to resize shit

34:17SPEAKERso that like Hibernate works or whatever.

34:19SPEAKERIt's one of the reasons I kind of washed out

34:21SPEAKERof tinkering with desktop Linux.

34:23SPEAKERGod, it's been almost a year now

34:24SPEAKERsince I was fooling with that.

34:25SPEAKERBut yes, like the Fedora installer, for example,

34:28SPEAKERmade way too many opinionated choices

34:30SPEAKERabout how to arrange the butter fest volumes

34:32SPEAKERin a way that kind of like defeated what I was trying to do

34:34SPEAKERand I kind of got fed up.

34:35SPEAKER- That's why you need to make Brad OS.

34:37SPEAKER- I guess so.

34:37SPEAKER- That's how you do this.

34:38SPEAKER- I guess so or I really should just run through Linux

34:40SPEAKERfrom scratch, but I'm pretty comfortable

34:42SPEAKERwith all this kind of manual partitioning

34:44SPEAKERand stuff at this point.

34:46SPEAKEROkay, so I had to do the manual Debian install,

34:48SPEAKERwhich meant that I had to like choose

34:50SPEAKERhow to configure my network and choose,

34:53SPEAKERmake the UEFI boot entries myself.

34:56SPEAKERLike you're kind of picking everything.

34:59SPEAKERAs I've said many times, there's like six ways, for example,

35:01SPEAKERto configure your network interface in Linux.

35:03SPEAKERSo you kind of just need to pick one and go with it.

35:06SPEAKER- Yeah, and that's a like no wrong choices situation.

35:10SPEAKER- Pretty much.

35:11SPEAKER- Like you can't, yeah.

35:13SPEAKER- Well, you know, this is a case where I would say,

35:15SPEAKERfor example, the Debian Handbook makes some recommendations

35:18SPEAKERabout like, how are you using this?

35:19SPEAKERIf you're on desktop, you should probably

35:21SPEAKERuse network manager.

35:22SPEAKERIf you're doing a server, system D network D

35:24SPEAKERmight be better, that's what I'm using.

35:25SPEAKEROr in that case, I think they describe it

35:28SPEAKERas the modern headless network configuration.

35:31SPEAKERSo like whatever.

35:32SPEAKER- Some people really don't like system D stuff, Brad.

35:33SPEAKERI don't know if you know that.

35:34SPEAKERBut that's a different topic for a different day, I think.

35:36SPEAKER- Yeah, it is.

35:37SPEAKEROkay, and then like I said, I'd have to maintain

35:40SPEAKERthe like the boot entries myself.

35:42SPEAKERWe've talked about it before.

35:43SPEAKERI use a ZFS specific boot loader

35:45SPEAKERthat maintains compatibility to make sure

35:47SPEAKERthat there's a whole--

35:49SPEAKER- Are you doing secure boot there too, or are you not?

35:52SPEAKER- Yes, I am doing secure boot there,

35:53SPEAKERwhich means that I have to,

35:54SPEAKERso I generated my own secure boot keys

35:56SPEAKERthat I enrolled into the UEFI.

35:58SPEAKERAnd I have to, so I have to sign,

36:00SPEAKERit's called ZFS boot menu is the boot loader I'm using.

36:02SPEAKERAnd yes, I have to sign every new version of that

36:05SPEAKERto make sure--

36:06SPEAKER- Do you do that manually, or do you have hooks

36:07SPEAKERthat do that when a new kernel gets pulled down?

36:09SPEAKER- ZFS boot menu, I do that manually.

36:11SPEAKER- Oh God.

36:12SPEAKER- But I set up DKMS, which is the,

36:16SPEAKERwe go again, did you know there are multiple subsystems

36:19SPEAKERin Linux for building kernel modules?

36:20SPEAKEROf course there are.

36:21SPEAKERThere's DKMS, which like Debbie uses, I think Arch uses,

36:25SPEAKERbut then there's AK mods,

36:26SPEAKERwhich is used by the Red Hat style distros.

36:30SPEAKERAnyway, those things can be, DKMS in particular

36:32SPEAKERcan definitely be configured automatically with your keys.

36:35SPEAKERSo anytime it builds a new ZFS module, for example,

36:38SPEAKERI have it set up to just sign it with my key automatically,

36:40SPEAKERand that's fine.

36:42SPEAKERI don't have to touch any of that stuff,

36:43SPEAKERbut the thing I was gonna say real quick,

36:45SPEAKERand I'm making it more complicated on myself

36:47SPEAKERby running ZFS root, to be clear, but--

36:49SPEAKER- Yeah, 'cause like theoretically,

36:50SPEAKERif you installed on like a butter FS or EX3 or something,

36:55SPEAKERyou'd set up the machine, do all this stuff,

36:59SPEAKERand then you'd flip on the ZFS side,

37:00SPEAKERyou wouldn't have to do any of this business.

37:02SPEAKER- I could totally be doing like an MD raid

37:04SPEAKERand EX4 setup for my boot volume,

37:07SPEAKERwhich is still mirrored and redundant,

37:09SPEAKERbut using like stuff that's in the kernel

37:10SPEAKERand not have to worry about it.

37:12SPEAKER- If you were doing this again, would you do that,

37:13SPEAKERor would you do the ZFS thing?

37:15SPEAKER- I've thought about moving to it.

37:17SPEAKERThe reason I cloned to ZFS is because I'm already using it

37:19SPEAKERto manage all the other volumes in this system,

37:22SPEAKERand I kinda just wanted to keep it all to one style

37:26SPEAKERof drive management for everything,

37:28SPEAKERbut I have thought about doing that before,

37:30SPEAKERand that's why I was so bummed that Bcache FS

37:32SPEAKERgot pulled out of the Linux kernel recently,

37:35SPEAKERbecause I was thinking maybe one day

37:36SPEAKERI'll move my boot volume to that,

37:38SPEAKERbecause that is in the kernel, but now it's not.

37:39SPEAKER- Scene drama.

37:40SPEAKER- Yes, kinda, yes, that's definitely some scene drama.

37:43SPEAKERTo give a very quick illustration

37:44SPEAKERof how this can get complicated,

37:47SPEAKERbecause ZFS is outside of the kernel,

37:49SPEAKERthey have to track kernel versions,

37:50SPEAKERmeaning like every version of ZFS they put out

37:52SPEAKERis compatible with only a range of kernels,

37:55SPEAKERtypically up to the newest kernel,

37:56SPEAKERbecause they are pretty good about staying up to date,

37:58SPEAKERbut occasionally, the point is occasionally

38:01SPEAKERa Linux kernel will ship before ZFS

38:03SPEAKERis ready to be compatible with it.

38:05SPEAKERSo it is possible, and this has happened to me before,

38:07SPEAKERit is possible to update your kernel beyond the point

38:09SPEAKERwhere the kernel can work with the file system

38:12SPEAKERyou've got your operating system on.

38:14SPEAKER- So then how do you fix that?

38:15SPEAKERYou boot off of a thumb drive and chroot into your system

38:18SPEAKERand change back to the old kernel?

38:19SPEAKER- Roll back a snapshot.

38:20SPEAKER- Okay.

38:21SPEAKER- Is typically how I've done it.

38:22SPEAKER- Now do you have snapshots set up for your ZFS snapshots

38:26SPEAKERset up that are integrated with your boot loaders,

38:28SPEAKERso you can just change back to the previous one?

38:30SPEAKER- So I think maybe that's a good way to focus this down

38:32SPEAKERso I don't ramble forever here,

38:33SPEAKERis to kinda go through the things TrueNest does for you

38:35SPEAKERand sort of talk about my equivalent.

38:37SPEAKER- Yeah, 'cause the nice thing about TrueNest

38:39SPEAKERis if you had a bad update,

38:40SPEAKERyou literally on the boot menu could just pull down

38:42SPEAKERthe last five states of the machine

38:44SPEAKERhowever many snapshots you had

38:46SPEAKERand roll back to how it was yesterday.

38:48SPEAKER- Yes, that's one of the things TrueNest manages for you

38:50SPEAKERvery well, is doing auto snapshots.

38:52SPEAKERYou do have to tell it to do that for some drives,

38:55SPEAKERbut the system drive in TrueNest,

38:57SPEAKERit just does snapshots every time you update.

39:01SPEAKERBut yes, that's one of the things I had set up on TrueNest

39:04SPEAKERwas just having it snapshot my,

39:06SPEAKERevery pool in the system gets a snapshot every hour,

39:09SPEAKERevery day, every week, et cetera,

39:10SPEAKERand there's policy for how often to call those.

39:14SPEAKERThose don't use any extra space to be clear.

39:16SPEAKERThat's kind of how copy on write file systems work.

39:18SPEAKERAnyway, I'm using a tool called Sanoid, S-A-N-O-I-D,

39:23SPEAKERwhich has been around for quite a while.

39:25SPEAKERIt was written by Jim Salter,

39:26SPEAKERwho is a sometime contributor to Ars Technica.

39:30SPEAKERI don't know if he still writes for them or not.

39:32SPEAKER- I don't know.

39:33SPEAKERI've always tried to avoid the noid.

39:34SPEAKERThat's my understanding,

39:37SPEAKERbut it seems like you've embraced the noid.

39:38SPEAKER- Yes, anyway, Sanoid is the most robust tool

39:42SPEAKERI've seen out there for ZFS auto snapshots.

39:45SPEAKERIt's command line only.

39:46SPEAKERIt is packaged for some distros,

39:49SPEAKERbut I just went and installed the DEB.

39:51SPEAKERIn fact, I think you build your own DEB

39:53SPEAKERfrom the GitHub repo and then install that.

39:55SPEAKER- And it triggers a snapshot at any time,

39:57SPEAKERlike you pull a package down

39:59SPEAKERfrom the Debian repository or something?

40:01SPEAKER- Yeah, yeah.

40:03SPEAKERAnd it's all policy driven.

40:05SPEAKERI should, it's probably worth noting here.

40:07SPEAKERIn my setup or a set up like this,

40:09SPEAKERunless you set up some kind of web UI,

40:11SPEAKERyou were just configuring everything

40:13SPEAKERfrom a terminal in text files and config files.

40:17SPEAKERBut yes, it's policy driven where you can tell it

40:19SPEAKERlike snapshot this pool every hour,

40:22SPEAKERkeep 48 hourly snapshots, et cetera, et cetera.

40:25SPEAKERYou know, by pool, you can configure that stuff.

40:29SPEAKERSanoid has an optional subcomponent called syncoid.

40:34SPEAKER- Oh. - S-Y-N-C, syncoid.

40:37SPEAKER- Okay. - For replicating

40:38SPEAKERthe snapshots that it's auto snapshotting to other volumes.

40:41SPEAKER- Oh, across the network or whatever.

40:43SPEAKER- Or locally. - Or locally, yeah, yeah.

40:45SPEAKER- It'll basically use any path over SSH or locally.

40:48SPEAKERSo you could have USB drives plugged in

40:50SPEAKERto replicate to if you wanted to.

40:51SPEAKEROr in my case, I swear to God,

40:53SPEAKERI'm gonna finish setting this up one day.

40:55SPEAKERIt's the Raspberry Pi down the hall

40:57SPEAKERwith the four drive enclosure bolted to it.

41:00SPEAKER- I'm gonna say, that's one of the things

41:02SPEAKERthat was one of the appealing features for me on ZFS

41:05SPEAKERwhen I first started down this path

41:07SPEAKERis that I could like set up another machine

41:09SPEAKERthat's plugged into, at the time, the tested office, right?

41:12SPEAKERAnd I just have an offsite backup of my stuff

41:16SPEAKERthat happens automatically.

41:17SPEAKER- Yes. - No, I don't bother

41:18SPEAKERwith that anymore. - Yeah, I mean,

41:19SPEAKERlike all of this, it is a lot of work.

41:21SPEAKERBut the workflow, typically the workflow

41:23SPEAKERthat I'm trying to achieve here

41:25SPEAKERis I never think about any of this stuff once it's set up.

41:27SPEAKERI just use the server as is.

41:29SPEAKERYou know, I put things on there like podcast recordings,

41:32SPEAKERgo to it, whatever.

41:33SPEAKERAt night, I've got it,

41:35SPEAKERor theoretically I will have it set up

41:37SPEAKERso that every night it just syncs

41:39SPEAKERwhatever that day's additions or changes are

41:41SPEAKERto the volume down the hall.

41:43SPEAKERAnd I basically never touch any of it

41:45SPEAKERunless I need a backup, right?

41:47SPEAKERThat's the goal, anyway.

41:51SPEAKERI've also got the snapshots on the NAS setup

41:54SPEAKERas what used to be called shadow copies in Windows.

41:57SPEAKERI think they've changed the name.

41:59SPEAKER- Shadow volume copies is what we used to call them.

42:00SPEAKERMaybe that's it.

42:02SPEAKEREffectively, I can get properties

42:04SPEAKERon any network-mounted folder

42:06SPEAKERand it'll show me a previous versions list

42:08SPEAKERand I can just roll back to...

42:10SPEAKERThat actually saved my ass a couple of weeks ago

42:12SPEAKERwhen I deleted some old ramble cast recordings

42:16SPEAKERjust to clear space.

42:17SPEAKERAnd then I can like...

42:18SPEAKERAnd that one is on a six hour call schedule.

42:22SPEAKERSo like two hours after I deleted them,

42:24SPEAKERthankfully I realized I needed something

42:26SPEAKERfrom one of those old recordings.

42:27SPEAKERAnd I was able to go into those,

42:29SPEAKERinto the previous version through the Windows File Explorer

42:31SPEAKERand pull that file out before it truly got eradicated.

42:34SPEAKER- That's good.

42:35SPEAKER- And now how am I sharing these things to Windows?

42:38SPEAKER- Yeah, I was gonna say like,

42:39SPEAKERwhat services are you running on this?

42:41SPEAKERSo that's the big question.

42:42SPEAKER- So I'll kind of keep going down the line here.

42:44SPEAKERSamba is SMB is what I'm using to share

42:47SPEAKERthe volumes on this thing over the network.

42:49SPEAKERThere are other options.

42:50SPEAKERThere's like NFS, which is a more traditionally

42:54SPEAKERUnix-y way to share files over.

42:57SPEAKER- Yeah, I use NFS for like Linux to Linux talking,

43:00SPEAKERbut not so much for Windows to Linux talking.

43:03SPEAKER- That's exactly why is the Windows support for NFS

43:06SPEAKERis very, very bad.

43:08SPEAKER- Yes, but it's garbage.

43:09SPEAKER- Bad is a true, that's a fine way to describe it.

43:12SPEAKER- Well, so the nice thing about NFS

43:15SPEAKERis it preserves permissions and it lets you map

43:17SPEAKERlike a user on one machine to a user on another machine

43:19SPEAKERor use active directory or something like that

43:21SPEAKERto talk across both of them, not active directory,

43:24SPEAKERbut like a shared source of truth

43:28SPEAKERfor which user is which user.

43:29SPEAKER- Yeah, in a more like natively Unix-y way, right?

43:32SPEAKERLike you can do that stuff with Samba as well,

43:34SPEAKERbut it's much more fiddly and arcane, I find.

43:37SPEAKER- With Samba, you usually crush down the permissions

43:39SPEAKERto be like the least common denominator

43:41SPEAKERthat everybody needs to be able to do

43:42SPEAKERthe thing you want to do.

43:43SPEAKER- Yeah, it's kind of a mess.

43:44SPEAKERAnyway, not a lot to say about Samba.

43:46SPEAKERLike I was using Samba or SMB in TrueNAS as well.

43:49SPEAKERAnd again, I do it because Windows is good with it.

43:52SPEAKERmacOS is also good with it so I can mount those volumes

43:55SPEAKERfor my MacBook as well.

43:56SPEAKER- It's funny, I often use AFP on my Linux machines

44:01SPEAKERto talk to the NAS when I'm on desktop

44:03SPEAKERbecause for whatever reason, like Nautilus

44:06SPEAKERand all the file managers that I've used

44:08SPEAKERin desktop Linux almost always see the AFP shares

44:12SPEAKERon my Synology NAS before they see the SMB shares.

44:17SPEAKER- Interesting.

44:19SPEAKER- So yeah, I don't know why that is.

44:21SPEAKERIt's just a weird like side effect

44:22SPEAKERof opening up the network browser

44:24SPEAKERand being like, oh, okay, so there's two things here.

44:26SPEAKERIt's the same machine both,

44:27SPEAKERbut this one always pops up faster, so I use it.

44:31SPEAKER- What was I gonna say about,

44:32SPEAKERoh, I've got all the advertisement turned off for that stuff

44:35SPEAKERso I always just mount stuff by IP.

44:37SPEAKER- Yeah, I have other people using my computer.

44:39SPEAKER- Yeah, I don't have that concern.

44:41SPEAKERSo Samba, not a lot to say there.

44:42SPEAKERYou configure it by managing smb.conf,

44:46SPEAKERlike it's relatively straightforward,

44:47SPEAKERalthough there's like a ton of complexity

44:50SPEAKERto advanced Samba configurations we don't need to get into.

44:53SPEAKER- Yep.

44:54SPEAKER- Okay, services and containerization.

44:57SPEAKER- Yeah.

44:58SPEAKER- Is like probably the thing that,

45:03SPEAKERI started to say probably the thing

45:04SPEAKERthat was the most problematic on TrueNAS.

45:06SPEAKERI don't think that's quite true,

45:07SPEAKERbut it was the thing that on BSD you only had jails for.

45:11SPEAKER- Well, and then they also had plugins,

45:13SPEAKERbut plugins were a little fraught.

45:14SPEAKER- Yeah, I never even touched the plugin stuff

45:16SPEAKER'cause people were like, oh, their Plex plugin

45:18SPEAKERis super outdated or doesn't work right or blah, blah, blah.

45:20SPEAKER- Yeah, the bigger problem with the plugins

45:22SPEAKERwas that they would always have

45:23SPEAKERsome sort of weird limitation.

45:25SPEAKERSo if you were doing something that was like,

45:29SPEAKERthat had relatively low overhead

45:32SPEAKERand relatively low interconnection with other services,

45:34SPEAKERit was usually fine.

45:36SPEAKERBut anytime you wanted to do anything

45:38SPEAKERthat went beyond like the default, then it was a problem.

45:41SPEAKERLike for example, the Plex plugin was great.

45:44SPEAKERFiguring out how to get your Plex database

45:46SPEAKERout of that Plex plugin,

45:48SPEAKERthe Plex database contains like the list of movies,

45:50SPEAKERthe number of times they've been watched,

45:53SPEAKERthe users that are allowed to attach to your server,

45:55SPEAKERall that kind of stuff,

45:56SPEAKERand a giant cache of all the art

46:00SPEAKERand the art that gets displayed in Plex

46:02SPEAKERaround each movie or TV show.

46:05SPEAKERFiguring out how to get that data out of the Plex plugin

46:07SPEAKERon FreeNAS was a nightmare.

46:09SPEAKER- I bet, man.

46:10SPEAKERI mean, it's not easy to get it out of a manual install.

46:12SPEAKER- No, it sucks.

46:13SPEAKER- Like if you've ever migrated Plex

46:14SPEAKERfrom one operating system to another,

46:16SPEAKERlike their support page for doing so is like 20 steps long.

46:20SPEAKERAnd you have to be kind of careful about it.

46:21SPEAKERHave you done it?

46:22SPEAKER- Yeah, I've done it three times.

46:23SPEAKERIt sucks every time.

46:24SPEAKER- They are very specific about like,

46:26SPEAKERokay, like before you log into the Plex on the new machine,

46:30SPEAKERyou have to log out of the one on the old machine,

46:32SPEAKERmake sure you don't clear the trash on that one though.

46:35SPEAKERWe copy these files over, like do this, do this.

46:37SPEAKERLike it's weirdly sort of fragile, it feels like.

46:41SPEAKER- So I have that set up now so that my Plex database

46:44SPEAKERis just as a folder, a network share on the NAS.

46:49SPEAKERAnd when I connect to it, I have to reverse engineer

46:53SPEAKERhow I did that every single time,

46:54SPEAKER'cause it's a huge pain in the butt.

46:55SPEAKER- It sure is.

46:56SPEAKEROkay, so my understanding is since TrueNAS has moved

47:00SPEAKERto Linux, I mean, obviously kind of like me moving

47:02SPEAKERto bare Linux that has given TrueNAS users

47:05SPEAKERlike way more options for the types of sort of containers

47:07SPEAKERand plugins they can use.

47:08SPEAKER- Are they still using jails?

47:10SPEAKER- Well, no, not on Linux.

47:13SPEAKER- Boy, that's a big topic.

47:14SPEAKER- So okay, so for folks who don't know,

47:17SPEAKERjails basically the short, short version

47:20SPEAKERof what a BSD jail is, we've talked about these before.

47:23SPEAKERIt's really cool, it's my favorite thing about BSD.

47:25SPEAKERIt's the reason I use TrueNAS for such a long time.

47:28SPEAKERIt basically lets you spin up a virtual,

47:31SPEAKERwhat looks like a virtual machine on your network

47:33SPEAKERthat's essentially, it has its own IP address,

47:35SPEAKERso you can give it its own like DNS entries,

47:38SPEAKERstuff like that, and it takes the system files

47:42SPEAKERfrom your main OS and lets you use them

47:45SPEAKERas if they were in a virtual machine

47:46SPEAKERwithout the overhead of a virtual machine,

47:48SPEAKERis kind of the short, short version.

47:50SPEAKER- So the reason for that is 'cause they're running

47:52SPEAKERon the same kernel as the rest of the machine is,

47:54SPEAKERso they don't use extra memory or there's,

47:57SPEAKERyou're not siloing resources off to them

48:00SPEAKERthat only they can access.

48:01SPEAKER- Yeah, that's the big one is that you're not,

48:04SPEAKERit's not like running a virtual machine where you say,

48:06SPEAKERoh, okay, I have four cores, eight cores total

48:08SPEAKERon this machine, I'm gonna devote two cores entirely

48:11SPEAKERto this VM, and then the main OS won't be able

48:13SPEAKERto use them anymore.

48:14SPEAKERIt let you resource share much more effectively.

48:16SPEAKER- It's a trade-off of lower resource overhead

48:18SPEAKERfor less security because a VM is much more sandboxed

48:21SPEAKERand less, is much more impenetrable than--

48:24SPEAKER- Yeah, it also means that you can access the file system

48:28SPEAKERa little bit easier, so you can, you basically you'd say,

48:31SPEAKERokay, I want these sub-volumes of your CFS array

48:33SPEAKERto be accessed by the jails, and for like a corporate user

48:39SPEAKERwho's really worried about security,

48:41SPEAKERmaybe not the best solution for a home person

48:42SPEAKERwho wants to make it easy for their Plex server

48:44SPEAKERto see their movies, piece of cake.

48:47SPEAKER- Yeah, so maybe I'll go in order

48:50SPEAKERof how I came to these things chronologically,

48:52SPEAKER'cause that might make it a little easier to explain

48:54SPEAKERand build on what we just talked about.

48:57SPEAKERSo in FreeBSD, I would say like jails are extremely secure

49:00SPEAKERbecause they are built in at the kernel level.

49:02SPEAKERLike they are what people would refer to

49:04SPEAKERas a first-class feature of the operating system,

49:06SPEAKERmeaning they've, you know, fully integrated into the kernel,

49:10SPEAKERlike fully security-audited, like I don't think

49:13SPEAKERa jail breach has ever been demonstrated before.

49:15SPEAKER- They've been around for a really long time,

49:17SPEAKERand they're battle-tested.

49:18SPEAKER- Yeah, it's 20-something years or whatever.

49:20SPEAKERLinux does not have an equivalent at that level.

49:24SPEAKER- They're built into the kernel.

49:25SPEAKER- Yes, like the Linux kernel itself

49:27SPEAKERdoes not expose a comprehensive containerization pattern,

49:32SPEAKERor whatever you wanna call it, in the way that FreeBSD does.

49:34SPEAKERNow, there have been Linux containers for like 15 years

49:37SPEAKERthat are built out of first-class Linux kernel features.

49:41SPEAKERLike, I'm getting into stuff I can kind of only

49:43SPEAKERsomewhat understand, like namespacing and control groups,

49:46SPEAKERwhich are explicit.

49:49SPEAKER- From a practical level, the solution that solves

49:52SPEAKERthe same problem in a different way on Linux is Docker, right?

49:55SPEAKER- Well, yeah, I mean, that's what I'm getting to,

49:56SPEAKERand that is largely what I have moved to.

49:58SPEAKERBut like I said, I'm going in order from, I kind of-

50:00SPEAKER- Oh.

50:01SPEAKER- When I got to the Linux NAS, I basically was trying

50:03SPEAKERto sort of replicate everything that TrueNAS did.

50:06SPEAKER- Yeah, it's a bad idea.

50:07SPEAKER- One-to-one, well, it was fine.

50:08SPEAKERIt worked fine for a while.

50:09SPEAKER- For a while.

50:10SPEAKER- Oh, dude, it always would have worked fine.

50:12SPEAKERIt was not like it was broken.

50:14SPEAKERIt was just that the management was ultimately,

50:18SPEAKERmuch like on BSD, effectively, you're managing

50:21SPEAKERa whole little OS for every service you're running.

50:24SPEAKER- I would argue that onerous management is not working fine.

50:27SPEAKER- Yes, yes, I mean, that's kind of where I got to,

50:28SPEAKEReventually, for sure.

50:29SPEAKERBut so the classic Linux container technology is LXC,

50:34SPEAKERor I guess Lexi, maybe, is how you're supposed

50:36SPEAKERto pronounce that, but it is three letters, L-X-C.

50:40SPEAKERLike that is the old school Linux container technology

50:44SPEAKERthat was kind of derived as a BSD jail equivalent

50:47SPEAKERout of kernel features in Linux.

50:50SPEAKERI used that for a long time, like two years, maybe,

50:54SPEAKERafter I moved to Linux.

50:55SPEAKERLike the tooling around that stuff is very bare bones.

50:58SPEAKERIt was a huge pain in the ass to administrate.

51:00SPEAKERRunning those containers in a secure way,

51:03SPEAKERlike a rootless way, 'cause I wanted to not run

51:05SPEAKERthose containers as root.

51:06SPEAKER- Yeah.

51:07SPEAKER- 'Cause if you're gonna be running services

51:08SPEAKERthat are exposed to the outside world,

51:09SPEAKERthat's a vector for--

51:11SPEAKER- Then you're giving those services root access

51:13SPEAKERto your machine, which was bad.

51:14SPEAKER- Right, like you don't, anything you want,

51:16SPEAKERanything that's running, pointing to the outside internet,

51:18SPEAKERin my book, you don't want running

51:20SPEAKERas any kind of privileged user,

51:21SPEAKERbecause if those things are compromised,

51:23SPEAKERthey get a lot of access to your machine all of a sudden.

51:26SPEAKERWithout getting into too much detail,

51:28SPEAKERthe LXC management was extremely fiddly.

51:32SPEAKERThe command line tools were super bare bones.

51:35SPEAKERYou kind of had to figure everything out yourself.

51:37SPEAKERRunning those containers rootless was a lot,

51:39SPEAKERalthough it did help me quite a bit for where I ended up,

51:43SPEAKERwhich is after a couple of years of that,

51:44SPEAKERI got tired of basically running,

51:46SPEAKERlike I was literally having to update

51:47SPEAKERa whole different Ubuntu install

51:51SPEAKERfor every service I was running.

51:53SPEAKER- I was gonna say, the thing I like about this

51:56SPEAKERis that using the LTS versions of,

51:58SPEAKERI'm using a LTS version of Ubuntu, not Debian,

52:00SPEAKERbut it means I don't have to update it very often, right?

52:04SPEAKERThe updates are relatively infrequent.

52:06SPEAKERThey're generally less scary than like a rolling distro,

52:10SPEAKERlike I'm running on my desktop.

52:12SPEAKERAnd for things that I use all the time

52:15SPEAKERand are important that they not break,

52:16SPEAKERit's really nice to log into that machine and be like,

52:19SPEAKERoh yeah, you don't have any packages to update.

52:21SPEAKERIf I had to do that for five Linux machines,

52:23SPEAKERI would kill myself.

52:24SPEAKER- I think at some point,

52:25SPEAKERI wrote a little tiny little wrapper script.

52:27SPEAKERI mean, we're talking like three lines or something

52:28SPEAKERto just update all of them at once,

52:31SPEAKERwhich is not difficult.

52:34SPEAKERI should point out like one cool thing though,

52:35SPEAKERabout that setup is on FreeBSD,

52:39SPEAKERthe jails are just on their jail of the same FreeBSD host

52:42SPEAKERthat you're running on.

52:43SPEAKERWith the Linux stuff,

52:44SPEAKERbecause there's so many different distros,

52:45SPEAKERI could be running Debian stable as the host,

52:48SPEAKERbut you can kind of pick any distro that they,

52:50SPEAKERand any version of a distro that they offer an image for

52:52SPEAKERas your containers.

52:54SPEAKERSo I was running like much newer Ubuntu server images

52:58SPEAKERas my container distros on top of,

53:02SPEAKERyou can totally do that, it's fine.

53:03SPEAKER- And you were able to do things like pass through access

53:05SPEAKERto quick sync and stuff like that.

53:07SPEAKER- Yeah, I mean that took work for sure.

53:09SPEAKERLike I had to learn more about Linux-

53:11SPEAKER- Dev/DRI.

53:13SPEAKER- Yes, yes, exactly.

53:14SPEAKERRender D128 was getting passed in.

53:17SPEAKERI thought, dude, this could be a two hour-

53:20SPEAKER- Render D is what?

53:22SPEAKER- Uh-huh, we could sit here for two hours

53:26SPEAKERtalking about this stuff.

53:27SPEAKERI had to get pretty well acquainted

53:28SPEAKERwith the way Linux handles hardware devices as files.

53:33SPEAKERTo do that, because again, LXC is like,

53:35SPEAKERit's well supported, but it's not well tooled,

53:39SPEAKERI guess is how I would put it.

53:40SPEAKERLike the tools again are not friendly at all.

53:42SPEAKERYou kind of have to figure everything out yourself.

53:43SPEAKERSo like-

53:44SPEAKER- Documentation good?

53:46SPEAKER- It's not nothing, but it's not enough.

53:50SPEAKER- Okay.

53:52SPEAKER- Okay, as a specific example,

53:54SPEAKERthe technique I ended up using

53:56SPEAKERfor passing the quick sync device into my Plex container,

53:59SPEAKERwhich I did all the work for

54:01SPEAKERbefore I found out that Plex requires a Plex pass

54:05SPEAKERto use hardware transcoding,

54:06SPEAKERalthough I was also running Jellyfin,

54:07SPEAKERso it was at least useful for that.

54:09SPEAKERThe solution that I ultimately used,

54:11SPEAKERwhich I was quite happy with for passing that quick sync

54:14SPEAKERdevice into the container is something I just dredged up

54:16SPEAKERfrom a random LXC forums post from three years before,

54:22SPEAKERto be clear.

54:23SPEAKERWhen you ask, is the documentation good?

54:24SPEAKERLike it's the bare bones stuff is good, but beyond that,

54:27SPEAKERand if you want to do anything advanced,

54:29SPEAKERyou're digging through forums.

54:30SPEAKER- Oh, great, okay.

54:31SPEAKERI don't like that for what it's worth.

54:33SPEAKER- Yeah, yeah, it's a lot.

54:35SPEAKER- So where'd you end up?

54:37SPEAKER- Where am I at now?

54:38SPEAKERI have, I've mostly moved off of LXC.

54:40SPEAKERI'm not running the LXC tools as such anymore.

54:44SPEAKERAt first, I can't remember what order I went in here,

54:46SPEAKERbut I'll just, let's just talk about Docker.

54:48SPEAKERThat's the thing everybody cares about here.

54:50SPEAKER- I mean, that's the right answer for this problem,

54:52SPEAKERit seems like most of the time.

54:53SPEAKER- Except I'm not running Docker.

54:56SPEAKER- What?

54:56SPEAKER- I'm using Podman to run Docker images.

54:59SPEAKER- Well, that's still running Docker.

55:01SPEAKER- Yes, it's still running Docker images.

55:02SPEAKERThe images are compatible.

55:04SPEAKERThey're two different tools that run the same things.

55:06SPEAKER- Yeah, is Podman a portainer?

55:10SPEAKERIt's like the, that's the,

55:13SPEAKERis that the lower overhead version of portainer?

55:17SPEAKERIs that a different thing?

55:18SPEAKER- Not at all, it's totally separate.

55:20SPEAKERPodman is, you could look at it

55:22SPEAKERas a different implementation of what Docker does.

55:24SPEAKERIt's a different container backend.

55:26SPEAKER- But you use like the same kind of Docker compose files

55:28SPEAKERor whatever?

55:28SPEAKER- No, oh, you could.

55:30SPEAKERThere is a Podman compose tool

55:32SPEAKERthat you can use effectively the same YAML files

55:35SPEAKERif you want it.

55:36SPEAKERBut I think I want to say even the developers

55:38SPEAKERof Podman compose basically say like,

55:40SPEAKERhey, this is here if you're coming from Docker

55:42SPEAKERand want to use the same style of YAML configuration

55:46SPEAKERyou're used to, but there are better ways to do that now.

55:51SPEAKERPodman came out of the Red Hat/System D/

55:55SPEAKERlike that whole constellation of products and services.

55:59SPEAKER- Okay.

56:00SPEAKER- Let's say like it's the standard Linux thing

56:02SPEAKERof somebody comes up with a way of doing something

56:05SPEAKERand then somebody else says, you know,

56:06SPEAKERI see X, Y, and Z problem with that.

56:08SPEAKERI think I can do it better.

56:10SPEAKERAlthough again, coming out of like the Red Hat world

56:12SPEAKERit is very well robustly supported and developed.

56:15SPEAKERLike it's not, this is not,

56:16SPEAKERPodman is not a fly by night thing at all.

56:19SPEAKERPodman and Docker are two different ways

56:20SPEAKERof doing the same thing,

56:22SPEAKERwhich is downloading images, quote unquote,

56:25SPEAKERof applications you want to run

56:27SPEAKERthat are fully self-contained sandboxed bundles

56:30SPEAKERof like the binary, the core binary of the service,

56:33SPEAKERall the libraries that are required to run it,

56:36SPEAKERyou know, kind of a-

56:37SPEAKER- Well, yeah, the idea is that it gives you

56:39SPEAKERthe machine state that you need

56:42SPEAKERto run the service that you want.

56:44SPEAKER- Now, ironically, it's doing effectively the same thing.

56:47SPEAKERAnd in fact, using the same kernel technologies by and large

56:50SPEAKERas the LXC containers I was doing.

56:52SPEAKER- It's just a friendlier interface on top of them.

56:53SPEAKER- Well, no, what it is, it's just bundling up

56:55SPEAKERonly exactly what that application needs to run,

56:58SPEAKERunless it's a poorly made image

56:59SPEAKERand somebody puts way too much stuff in there.

57:00SPEAKERBut by and large, like well authored images

57:04SPEAKERand people on the Linux channel on our server

57:07SPEAKERare definitely going to scream at me at some point

57:09SPEAKERin this conversation, but-

57:10SPEAKER- That's fine.

57:11SPEAKER- My understanding of it in the little bit

57:12SPEAKERI've looked at how you build up Docker images

57:14SPEAKERis like you really only want to put exactly as much

57:17SPEAKERas you need for the image to work and nothing more in there.

57:20SPEAKER- Yeah, that's right.

57:21SPEAKERSo like Docker images are also based

57:24SPEAKERon specific Linux distros typically,

57:26SPEAKERbut they're so stripped down

57:27SPEAKERthat they're not full Linux distros.

57:29SPEAKERMy understanding is you could run a full Linux distro

57:31SPEAKERinside a Docker container if you wanted.

57:32SPEAKER- Yeah, people run like, you can run like a desktop,

57:37SPEAKERyou can run a full desktop inside a Docker container

57:39SPEAKERif you want, but it's like, I wouldn't do that.

57:41SPEAKER- What is, there is a distro box,

57:44SPEAKERI think is the tool that I see recommended.

57:46SPEAKERThere is a tool that can use either Docker or Podman

57:49SPEAKERas its backend that will let you run.

57:52SPEAKERYes, it is distro box.

57:54SPEAKERLike you can try out other distros

57:57SPEAKERin a Docker slash Podman container type format.

58:01SPEAKERThat's yet another way to do the thing I was doing before.

58:04SPEAKER- So what are you running in all these Podman's?

58:06SPEAKER- It's typical media.

58:07SPEAKERSo I actually don't run that many.

58:09SPEAKERIt's typical like media service stuff,

58:11SPEAKERlike Plex, Jellyfin, OwnTone.

58:14SPEAKER- OwnTone is your MP3 server?

58:16SPEAKER- Well, it's the AirPlay server.

58:18SPEAKERIt's the thing that exposes to the network as an AirPlay.

58:22SPEAKERLike what are the products they make that can do this?

58:25SPEAKERLike back in the day, it would be like your iTunes

58:28SPEAKERrunning on your Mac would show up as a library to play from.

58:31SPEAKERBut these days it's more like what?

58:34SPEAKER- Like what Plex Music does that?

58:37SPEAKER- Or I mean from Apple though,

58:38SPEAKERlike does the HomePod do that?

58:39SPEAKERIs that what the HomePod is for?

58:41SPEAKER- You just ask it to play something and it plays it.

58:43SPEAKERYou don't have to think about it.

58:44SPEAKER- Well, at any rate OwnTone is a way

58:46SPEAKERto look like you have an Apple, like an iTunes library

58:49SPEAKERon your, because I have a lot of AirPlay speakers

58:50SPEAKERin the house.

58:52SPEAKER- So it's funny, like the AirPlay speakers

58:54SPEAKERthat you're talking about are not current Air,

58:56SPEAKERlike not the AirPlay speakers from the old times.

59:00SPEAKERThe thing you're talking about is that iTunes library

59:02SPEAKERsharing thing from like 2008,

59:05SPEAKERwhere one day you turned on iTunes and all of a sudden

59:08SPEAKERyou saw everybody else's iTunes in the office

59:10SPEAKERand then you could steal MP3s from them.

59:12SPEAKER- Well, could you actually really get the files

59:14SPEAKERor just stream them?

59:15SPEAKER- Yeah, do it right if you get the files.

59:17SPEAKER- Of course you could, of course you could.

59:19SPEAKER- Yeah, but, okay, so.

59:22SPEAKER- Yeah, so, but real quick, I mean,

59:25SPEAKERdo we want to get into why I'm using Podman, so Docker?

59:28SPEAKER- No, it's fine, it's better.

59:31SPEAKER- I mean, that's a big thorny question.

59:32SPEAKER- Just say one way or the other, it's a yes or no, Brad.

59:34SPEAKER- I like it better, but I've barely touched Docker.

59:37SPEAKERI have literally used Docker for about 30 minutes ever.

59:39SPEAKERLike, I just don't have a good frame of reference,

59:41SPEAKERbut the reason my understanding is the reason Podman

59:44SPEAKERcame to be is because the people who made Podman,

59:47SPEAKERagain, people in that Reddit, not Reddit, Jesus,

59:51SPEAKERmy brain is so poisoned.

59:52SPEAKERThe Red Hat, like system D kind of world,

59:54SPEAKERlooked at Docker and felt they were not,

59:56SPEAKERDocker was not taking it seriously enough

59:58SPEAKERto let you run containers, not as root.

1:00:00SPEAKER'Cause again, like we said, running services as root

1:00:03SPEAKERis dodgy from a security perspective.

1:00:06SPEAKER- Questioning a lot of my choices right now,

1:00:08SPEAKERjust for the record.

1:00:08SPEAKER- Well, Docker has since, my understanding finally,

1:00:12SPEAKERbelatedly got around to more or less fixing that problem.

1:00:15SPEAKERBut I think I'm really speaking out of turn here.

1:00:18SPEAKERI think out of the box,

1:00:19SPEAKERit's still probably running everything as root,

1:00:21SPEAKERunless you go out of your way to change that.

1:00:23SPEAKERBut don't quote me on that.

1:00:25SPEAKER- I don't know about that.

1:00:26SPEAKERI don't actually pay attention to that usually.

1:00:28SPEAKER- Anyway, very briefly, Podman is basically built

1:00:30SPEAKERfrom the ground up to let you run containers

1:00:32SPEAKERas unprivileged users where Docker kind of,

1:00:35SPEAKERI think that had to be retrofitted on.

1:00:38SPEAKER- Okay, that makes sense.

1:00:39SPEAKER- Is basically the selling point.

1:00:40SPEAKERPodman also doesn't run a daemon,

1:00:41SPEAKERmeaning like a monitoring process,

1:00:44SPEAKERsupervising process in the background.

1:00:46SPEAKER- Did you say daemon?

1:00:47SPEAKER- Yes, I did.

1:00:48SPEAKER- Don't you mean daemon?

1:00:49SPEAKER- No, I don't.

1:00:50SPEAKER- It says D-A-E-M-O-N right here.

1:00:51SPEAKER- That's right.

1:00:52SPEAKERLook it up in the dictionary, man.

1:00:53SPEAKER- Daemon.

1:00:54SPEAKER- Not to appeal to authority here.

1:00:55SPEAKER- Opposite of the night man.

1:00:57SPEAKER- Not to just appeal straight to the dictionary authority,

1:01:00SPEAKERbut if you look it up, it says archaic spelling of daemon.

1:01:07SPEAKER- I don't know how I feel about this,

1:01:08SPEAKERbut I'm gonna let it go.

1:01:09SPEAKERI'm gonna let the chat handle this.

1:01:11SPEAKERDude, the free BSD mascot is a devil.

1:01:14SPEAKER- Yeah.

1:01:15SPEAKER- There's a reason for that, I'm pretty sure.

1:01:18SPEAKER- Daemon.

1:01:19SPEAKER- Anyway, whatever, whatever.

1:01:21SPEAKERPodman doesn't run like a supervisory process

1:01:23SPEAKERthat oversees everything else, like Docker, security stuff.

1:01:29SPEAKERSo I run all of my containers, all my Podman containers

1:01:33SPEAKERunder a kind of a nobody, unprivileged user.

1:01:37SPEAKER- Okay.

1:01:37SPEAKER- And then we don't really need to get down this road

1:01:39SPEAKERtoo much, but I'm actually using a setting

1:01:42SPEAKERwhere the actual container processes inside the containers

1:01:44SPEAKERdon't even run as that user.

1:01:46SPEAKERThey run as a subordinate six digit user ID,

1:01:50SPEAKERlike ephemeral user ID.

1:01:52SPEAKERLXC does the same thing.

1:01:53SPEAKERThis is actually pretty standard Linux stuff.

1:01:55SPEAKERSo all of my Plex process and my own tone process

1:01:59SPEAKERare actually running as user 100999.

1:02:03SPEAKER- How do they have access to the,

1:02:05SPEAKERdo you have to give that user access

1:02:06SPEAKERto your files in the file system?

1:02:08SPEAKER- Okay.

1:02:09SPEAKER- ACLs, do you remember when we interviewed Jeremy Allison,

1:02:12SPEAKERI believe was the Samba maintainer, co-founder, remember?

1:02:15SPEAKERRemember that?

1:02:16SPEAKERRemember him talking about ACLs?

1:02:17SPEAKER- Mm-hmm.

1:02:18SPEAKER- So I had always-

1:02:20SPEAKER- He called them ACLs?

1:02:21SPEAKER- He called them ACLs and explained a lot.

1:02:23SPEAKERYou know, he's been around a while.

1:02:25SPEAKERHe's got more authority in this domain than I do.

1:02:27SPEAKER- Yeah.

1:02:28SPEAKER- I always said them ACLs, but ACLs are,

1:02:30SPEAKERand this might be something that comes up

1:02:31SPEAKERon the dual boot and diaries at some point,

1:02:33SPEAKERaccess control lists.

1:02:34SPEAKERThey're ways to give files, other subordinate permissions

1:02:38SPEAKERto the main ownership permissions.

1:02:41SPEAKER- Oh yeah, it's like the advanced permission stuff

1:02:44SPEAKERin Windows NT.

1:02:45SPEAKER- Yeah, like Windows NTFS has a similar,

1:02:47SPEAKERin fact, NTFS has ACLs, like it's totally a cross-end.

1:02:51SPEAKERIt's, anyway, so I use ACLs to basically just give that,

1:02:55SPEAKERgive that six digit UID access to the directories it needs

1:02:59SPEAKERand nothing else.

1:03:00SPEAKER- Oh, see, I just crunch everything down to user 1000

1:03:02SPEAKERand share it across the,

1:03:03SPEAKER'cause I'm sharing across the network,

1:03:05SPEAKERso I'm connecting across the network,

1:03:06SPEAKERso I'm, it gets really weird and complicated

1:03:08SPEAKERif you don't do that.

1:03:09SPEAKER- Yeah, to very briefly defend

1:03:11SPEAKERmy kind of security maximalists position,

1:03:15SPEAKERA, this machine's got like my whole kind of digital life

1:03:18SPEAKERon it from college on.

1:03:20SPEAKERB, theoretically, I am once again going to point it

1:03:24SPEAKERat the internet at some point.

1:03:25SPEAKERLike, you know, I ran a Minecraft server

1:03:27SPEAKERand a little web server on it for a while.

1:03:28SPEAKERThose things are not currently running,

1:03:30SPEAKERbut I want to start using it for game servers

1:03:33SPEAKERand like a blog and stuff again at some point.

1:03:35SPEAKERAnd I, to be real, like I'm just enough

1:03:39SPEAKERof a minor public figure and have had just enough contact

1:03:42SPEAKERwith anonymous trolls in the past that I'm pretty,

1:03:46SPEAKERI'm pretty a little bit of paranoia,

1:03:48SPEAKERI feel like it was a long way here.

1:03:49SPEAKER- That's why you have to VPN into my house

1:03:51SPEAKERto use my net servers now.

1:03:53SPEAKER- Like so, so like the point I'm making

1:03:55SPEAKERis the average person maybe does not need to be quite

1:03:57SPEAKERas stringent about containerization

1:04:00SPEAKERand siloing things off as I am.

1:04:04SPEAKER- I think my solution for that problem

1:04:05SPEAKERis to just put everything on a different machine, right?

1:04:08SPEAKER- Sure, yes, that is not quite air-gapping,

1:04:13SPEAKERbut that's certainly a more physical separation

1:04:15SPEAKERthan having all the storage in the same machine

1:04:17SPEAKERthat all the services are running on.

1:04:19SPEAKER- Yeah, 'cause with the outward-facing services running on,

1:04:23SPEAKERso I have the two machines, I have the NAS

1:04:25SPEAKERand then I have the little B-Link

1:04:27SPEAKERwith the Alder Lake Celeron in it.

1:04:30SPEAKERAnd basically I do least valuable permissions

1:04:36SPEAKERfor each of the services.

1:04:38SPEAKERSo anything that lives outside my network

1:04:41SPEAKERdoesn't get right permission to the NAS

1:04:43SPEAKERand I just enforce that via the SMB shares

1:04:46SPEAKERor NFS shares, whatever I use.

1:04:48SPEAKER- So even if one of the services gets compromised,

1:04:50SPEAKERit still only has access to whatever network shares

1:04:53SPEAKERare mounted from there and not the entire system?

1:04:55SPEAKERYes. - Exactly.

1:04:57SPEAKER- Yes, indeed.

1:04:58SPEAKER- Okay, so Podman instead of Docker

1:05:00SPEAKERfor application service containers.

1:05:02SPEAKERI still do use LXC containers off and on, not as much.

1:05:07SPEAKER- They should have called those Podman containers,

1:05:08SPEAKERPodcons, just for the record, I think.

1:05:11SPEAKERThat's my, or Podcans, maybe, I don't know.

1:05:14SPEAKER- Podcans, Pod. - Yeah.

1:05:16SPEAKER- Podman.

1:05:17SPEAKER- Yeah, it sounds like a kind of a crappy

1:05:20SPEAKERB-tier Marvel superhero.

1:05:22SPEAKER- Kind of, kind of.

1:05:24SPEAKERSo I, like I said, I still use the occasional LXC,

1:05:27SPEAKERlike full distro OS container.

1:05:30SPEAKERWhat I'm now using for that is a tool called Incus.

1:05:32SPEAKERI-N-C-U-S.

1:05:35SPEAKER- Is that based on the demon incubus?

1:05:37SPEAKER- No, it's a type of cloud.

1:05:39SPEAKERI believe the cumulonimbus incus.

1:05:43SPEAKER- Oh, okay.

1:05:44SPEAKER- It's the anvil cloud, actually.

1:05:45SPEAKERJust looked this up the other day.

1:05:46SPEAKERDo you know, have you seen those clouds that form

1:05:48SPEAKERand look like an anvil?

1:05:49SPEAKER- No.

1:05:50SPEAKER- You should, you should Google.

1:05:52SPEAKERYeah, cumulonimbus incus is the type of cloud.

1:05:54SPEAKERIt's like, it's wild looking.

1:05:56SPEAKERI've never, certainly never seen one of these in person.

1:05:58SPEAKER- Oh, yeah, I've never seen one of these in person.

1:06:00SPEAKER- I think I have read about them.

1:06:01SPEAKERIt's like a type of cloud that like levels off on top

1:06:03SPEAKERand just looks like a literal anvil.

1:06:05SPEAKERIt's kind of crazy.

1:06:06SPEAKER- Phil Plait occasionally posts pictures of clouds

1:06:08SPEAKER'cause he lives in Colorado where they have good cloud.

1:06:11SPEAKER- Yes.

1:06:12SPEAKER- And, yeah, okay.

1:06:12SPEAKER- So what Incus is is actually a fork of LXD.

1:06:16SPEAKERHave you ever heard of that?

1:06:17SPEAKER- Yeah, I think we talked about it a little while ago

1:06:19SPEAKERand I said LXD's what and then you made a face at me.

1:06:22SPEAKER- We sure did.

1:06:23SPEAKERLXD came out of Canonical, the Ubuntu people.

1:06:25SPEAKER- Yeah.

1:06:26SPEAKER- It's basically a more advanced way

1:06:28SPEAKERof managing both LXD containers and VMs, virtual machines.

1:06:32SPEAKER- Okay.

1:06:33SPEAKER- Like actual virtual machines.

1:06:34SPEAKERIt's just using QEMU to run virtual machines under the hood.

1:06:38SPEAKERI'm sure you're familiar with that.

1:06:40SPEAKER- Oh, yeah.

1:06:41SPEAKER- I know you guys have talked about it.

1:06:42SPEAKER- I'm gonna ask a question though, just so I know,

1:06:44SPEAKERso the audience knows who doesn't know.

1:06:45SPEAKER- Okay.

1:06:46SPEAKER- Hey, Brad, what's QEMU?

1:06:47SPEAKER- It's lower level virtual machine management technology

1:06:51SPEAKERat Linux that hooks into KVM, which is the kernel,

1:06:54SPEAKERthe kernel virtual machine hypervisor.

1:06:57SPEAKER- Okay.

1:06:58SPEAKER- Yes.

1:06:59SPEAKER- That's cool.

1:06:59SPEAKERI haven't actually used that.

1:07:00SPEAKERI use bottles and stuff.

1:07:01SPEAKERI don't use virtual machines.

1:07:02SPEAKER- I swear I've heard Adam talk about QEMU.

1:07:04SPEAKER- Adam talked about different ways

1:07:06SPEAKERto manage virtual machines a couple of times, but yeah.

1:07:08SPEAKER- And then some of those front ends,

1:07:09SPEAKERI don't know if bottles is like this,

1:07:11SPEAKERbut some of those front ends that you're talking about

1:07:12SPEAKERon Duluth Diaries will also be something

1:07:15SPEAKERthat sits on top of something like QEMU.

1:07:17SPEAKERIn fact, that's what Incus is.

1:07:18SPEAKERLike it's all it's doing.

1:07:19SPEAKERIncus is just a slightly friendlier front end

1:07:22SPEAKERto both LXC and QEMU for VMs.

1:07:25SPEAKER- Bottles is just an easier way to package up wine MVP

1:07:29SPEAKERfor running wine apps.

1:07:31SPEAKER- That's a totally separate thing.

1:07:32SPEAKERAnyway, anyway, Canonical still makes LXD,

1:07:35SPEAKERbut there was other scene drama

1:07:36SPEAKERand the original LXD developers quit the company

1:07:38SPEAKERand forked it and it became Incus.

1:07:41SPEAKER- Okay.

1:07:42SPEAKER- So really the only point of that is it's kind of nice

1:07:44SPEAKERto manage your system containers and VMs

1:07:46SPEAKERfrom the same common command line interface.

1:07:48SPEAKER- It makes sense, yeah.

1:07:49SPEAKER- The syntax is all the same.

1:07:50SPEAKERIt's quite easy to manage.

1:07:51SPEAKERThat's not even remotely the only way

1:07:53SPEAKERto do that exact thing though.

1:07:55SPEAKERStop me if you've heard this one before.

1:07:57SPEAKER- I mean, look, there's an infinite number of ways

1:07:58SPEAKERto do pretty much everything in Linux, man.

1:08:00SPEAKER- Yeah, there's stuff like libvert

1:08:02SPEAKERthat I think is probably actually more popular

1:08:03SPEAKERthan Incus that does the same thing

1:08:06SPEAKERand a lot of other stuff at the same time.

1:08:07SPEAKERAnyway.

1:08:08SPEAKER- Now you're reaching the part of the list

1:08:09SPEAKERwhere it's like, there's just a series of letters

1:08:12SPEAKERthat don't, that like, hey, here's the first four letters

1:08:16SPEAKERon the A row of the keyboard

1:08:17SPEAKERand then four letters that look like a word, but aren't.

1:08:21SPEAKERDo you know how hard it was to get Google Docs

1:08:23SPEAKERto let me just type ASDF into this bullet point

1:08:25SPEAKERand not have it autocorrect something else?

1:08:27SPEAKER- Look, I just, the new Google Docs autocorrect blows.

1:08:31SPEAKERThey added AI to it and it's the worst thing ever.

1:08:34SPEAKER- Notepad was autocorrecting my typing this morning.

1:08:37SPEAKERI had to dig into the settings for that and turn that off.

1:08:39SPEAKERWhat are they doing?

1:08:40SPEAKER- I'm increasingly on the,

1:08:42SPEAKERmaybe I should just go back to one of the simple note clones

1:08:45SPEAKERor like an Obsidian branch or one of those.

1:08:47SPEAKER- We should do, I installed the Obsidian app.

1:08:50SPEAKERI have not like spent a lot of time on that,

1:08:53SPEAKERbut I want to start, I need to get off a simple note

1:08:56SPEAKERand want to start syncing text files to something I control.

1:08:59SPEAKERAnd so I need to put some time into like self-hosting

1:09:03SPEAKERor putting it in my VPS or something.

1:09:05SPEAKER- The problem is I've started,

1:09:06SPEAKERI've been using Notion now for like three years

1:09:09SPEAKERand the Wiki structure of that is it turns out really good

1:09:12SPEAKERfor like complex branching, many faceted notes

1:09:17SPEAKERlike I like to make and Obsidian

1:09:20SPEAKERjust can't, Obsidian is always like one layer, it sucks.

1:09:23SPEAKER- That's like a proprietary Notion feature, right?

1:09:25SPEAKERPretty much, or you can't just port that

1:09:27SPEAKERto some other application, right?

1:09:29SPEAKER- I'm sure like you could, but I'm sure somebody's done it,

1:09:32SPEAKERbut it's not in like mainline Obsidian for sure.

1:09:34SPEAKER- Sure, okay, so very briefly here,

1:09:36SPEAKERI use a tool called ASDF, which is a multi runtime manager.

1:09:41SPEAKERThis is something that if you're running a server,

1:09:43SPEAKERif you're running like a big NAS like I'm doing

1:09:45SPEAKERand it's just Linux, like this might be something

1:09:47SPEAKERthat would come in handy for you.

1:09:50SPEAKERWhat it is, is a way of installing runtimes

1:09:52SPEAKERfor kind of every programming language you can think of,

1:09:55SPEAKERPython, Rust, Ruby, Go, a bajillion others.

1:09:59SPEAKERThere's another tool out there now.

1:10:01SPEAKER- Why not just do that in the system packages?

1:10:04SPEAKER- So that's something you will discover

1:10:05SPEAKERif you start tinkering with that stuff quickly

1:10:07SPEAKERin your Linux distro is that you really do not want

1:10:10SPEAKERto mess around with the system Python,

1:10:14SPEAKERthe system X, Y, or Z, the system Rust,

1:10:17SPEAKERthe system, it might vary by language,

1:10:19SPEAKERbut Python in particular is one where you will constantly,

1:10:21SPEAKERin fact, I think PIP, when you try to install Python

1:10:24SPEAKERpackages with PIP, but with the system installed version

1:10:27SPEAKERof Python, it'll straight up warn you like,

1:10:29SPEAKERhey, you should not be installing random packages

1:10:33SPEAKERinto your system Python, you should use

1:10:35SPEAKERsome kind of runtime manager.

1:10:36SPEAKER- Because the different versions of Python

1:10:38SPEAKERaren't compatible with each other is the main reason.

1:10:40SPEAKER- I'm not well versed enough to know exactly

1:10:43SPEAKERwhat all the risks are there.

1:10:44SPEAKERBut anyway, I use ASDF, there's another tool

1:10:47SPEAKERthat I think is getting more popular now

1:10:49SPEAKERcalled MISE, M-I-S-E.

1:10:52SPEAKER- Okay.

1:10:52SPEAKER- As in Mise en place.

1:10:54SPEAKER- Oh no.

1:10:54SPEAKER- Yes.

1:10:55SPEAKERLook, man, I get it.

1:10:57SPEAKER- Look, naming stuff's hard, I'm not gonna judge.

1:10:59SPEAKER- Naming an open source project

1:11:00SPEAKERis probably pretty difficult, but I would probably

1:11:02SPEAKERbe tempted to just name it like thing D at this point.

1:11:05SPEAKER- I like--

1:11:06SPEAKER- It's like, whatever it is, D.

1:11:08SPEAKER- Look, I have a lot of the perfect respect

1:11:10SPEAKERfor the person who's like, fuck it,

1:11:11SPEAKERI'm just gonna call it ASDF 'cause that's the first

1:11:13SPEAKERfour letters my fingers hit on the keyboard.

1:11:15SPEAKER- I think it is kind of amazing that nobody

1:11:17SPEAKERever called a piece of software ASDF until now.

1:11:20SPEAKER- Yeah, it's like, yeah, man, that's pretty good, okay.

1:11:24SPEAKER- It's pretty good.

1:11:25SPEAKERAnyway, that might be something to look at

1:11:28SPEAKERif you're going to run a bunch of little random services

1:11:31SPEAKERlike I do in different languages.

1:11:33SPEAKERIt's nice to be able to just be like,

1:11:34SPEAKERhey, I just want to install a Ruby runtime

1:11:36SPEAKERand then use the gem package manager

1:11:38SPEAKERthat comes with Ruby to install this little utility.

1:11:41SPEAKER- Typically, when I want to do something like that,

1:11:42SPEAKERI typically just grab a Docker container and do it in that.

1:11:45SPEAKER- Yeah, yeah, if that's provided.

1:11:47SPEAKER- That handles the business for me.

1:11:49SPEAKERThe benefit of using Docker, I think over,

1:11:51SPEAKERI don't know what the kind of,

1:11:55SPEAKERlike how many people are building packages for Podman,

1:12:00SPEAKERbut like you go to Docker--

1:12:02SPEAKER- So actually, Podman images are the same as Docker images.

1:12:05SPEAKERThere's no distinction there in terms of creating,

1:12:08SPEAKERyou don't make a Docker image and a Podman image.

1:12:10SPEAKERI probably should have specified that.

1:12:12SPEAKERPodman is actually a command line compatible,

1:12:15SPEAKERwhere they, especially when it was new,

1:12:16SPEAKERthey were just straight up like,

1:12:17SPEAKERyou can just alias Podman to Docker if you want

1:12:19SPEAKERand keep running the same Docker commands

1:12:21SPEAKERyou've been running.

1:12:22SPEAKERIt uses the same images, runs the same commands.

1:12:24SPEAKERIt's just a different backend,

1:12:26SPEAKERbut in terms of the user experience,

1:12:29SPEAKERpretty much the same thing.

1:12:30SPEAKER- So if you go to like one of the Docker,

1:12:32SPEAKERone of the sites that has the like big lists

1:12:33SPEAKERof Docker images that people have built,

1:12:35SPEAKERyou just grab the, wow, that's amazing.

1:12:37SPEAKER- Half of the Podman or half of the images

1:12:39SPEAKERI'm running with Podman are straight off of like Docker.io.

1:12:42SPEAKER- Oh, that's rad.

1:12:43SPEAKER- They're all the exact same images.

1:12:44SPEAKERActually, sorry, there was one other detail about Podman

1:12:46SPEAKERthat you will appreciate that I really do need to mention,

1:12:48SPEAKERwhich I think is like the reason to use it now,

1:12:50SPEAKERbesides the root list stuff,

1:12:52SPEAKERwhich is that the more modern versions of Podman

1:12:54SPEAKERintegrate very directly with systemd.

1:12:57SPEAKER- Oh.

1:12:57SPEAKER- So I don't actually touch the Podman

1:13:00SPEAKERcommand line interface at all anymore.

1:13:02SPEAKER- So you just started from a systemd,

1:13:03SPEAKERso you make a systemd file that launches them?

1:13:06SPEAKER- Yes, if you're familiar

1:13:07SPEAKERwith what systemd unit files look like,

1:13:09SPEAKERlike a .service file, .timer, .mount,

1:13:12SPEAKERthere's a million types of systemd units,

1:13:14SPEAKERbut you write a .container file with the same syntax

1:13:18SPEAKERthat you would write like a service file with,

1:13:20SPEAKERand then-- - Oh, that's wild.

1:13:22SPEAKER- And then it auto-generates a service

1:13:23SPEAKERoff of that .container file,

1:13:26SPEAKERand you never-- - That runs the Docker image.

1:13:27SPEAKER- So you never even touch the Podman thing at all,

1:13:30SPEAKERlike the command line stuff, tools at all,

1:13:32SPEAKERyou just start and stop those services

1:13:34SPEAKERlike you would every other systemd service.

1:13:37SPEAKER- Wow, so that's, at some point in this year,

1:13:40SPEAKERI hope the Wildcat Lake CPUs will launch

1:13:44SPEAKERand computers will still be inexpensive enough

1:13:46SPEAKERthat I can afford to buy one of them,

1:13:48SPEAKERand I'm gonna move my little b-link server