Nice! Great to see innovation in the Jekyll space.
Quick FYI on Jekyll performance:
I noticed slow generation times on one of my sites and traced it to a plugin that was spawning a git process to get the last commit info for every page. I wrote a drop-in replacement called jekyll-last-commit[0] that uses the ruby libgit2 wrapper for improved performance. Details on its origins are in an old HN comment[1] if you are interested!
The other last-modified-plugin is now archived, and no longer included in the default Jekyll, but the sitemap and seo plugin still seem to rely on the page.last_modified_at property, albeit undocumented (https://github.com/jekyll/jekyll/issues/9702), so might be worth getting the implementation switched upstream perhaps?
We show the last-modified date on endoflife.date products (https://endoflife.date/jekyll), but I'm not sure how we have the property set right now, since I don't remember including either plugin. Thanks for creating this, I have a few other sites where this will be quite helpful.
Although I don’t use Jekyll, loved this idea. I think using SQLite as the main data source would be a nice way to preserve content and play around with different static generators if they all had a plugin like this.
Also TIL about the baked data pattern, which I think is exactly what I needed for an upcoming project, so thanks for that. Though, I do align with one of the commenters here—this doesn’t seem like the same thing as the Baked data pattern in that Simon’s approach was about using a server rendered app with read-only data instead of generating a lot of static pages.
Nevertheless, this seems like a nice way for static generators to work with bit more complex data sources without dumbing them down to JSON/YAML.
I'm a little confused. The baked-data model is so you DON'T have to generate a thousand static pages. But this solution does exactly that.
Not complaining, mind you. My kid is trying to learn HTML/CSS/JS and wants to put together a read-only website with a database backend. I'll be pointing him this way as an ootion once he's far enough.
But it's still puzzling to link it to baked-data. Maybe I'm missing something.
> bundling a read-only copy of your data alongside the code for your application, as part of the same deployment
You can see https://github.com/captn3m0/northwind for example, which bundles the entire database alongside the code in the _db/northwind.db file. While Simon considers it primarily for dynamic apps, you have the ability to build PWAs and other interesting apps with the baked data pattern.
This is awesome, and would solve an actual problem I've been thinking through... if only I had selected Jekyll for my static site generator. Does anyone know of a similar solution for Hugo?
Technically I'm sure I could run a script to generate .md (or a format like .csv that Hugo can work with) from my database, but this seems like it might be easier for a database that updates frequently.
Nice to see this project here! I've been using it with the Steam API to publish a list of the games I've been playing on my personal website: https://yboulkaid.com/games
I found that separating data from content on a Jekyll site is a really powerful way to have anything from photo galleries, blog entries, book lists, easily changeable menus etc...
This is so cool, bookmarked it. It sort of get the impression that Jekyll is slowing turning away from being a boring static site generator to something way interesting.
When you went from Jekyll to Kubernetes, it was must have been like when Ted Kaczynski first learned about battery-operated timed fuses.
Enjoyed the read.
Looking forward to when you chuck the whole thing into the bin and run Wordpress on your linux home server, fronted by a free Cloudflare zero trust tunnel.
I'm mostly focusing on sqlite-as-the-data-source, but I can imagine something like this might be in scope for a project like lunr.js[1], which currently uses a large index served as JS/JSON. You can write a Jekyll plugin in a few lines of ruby code (with no gem management needed, just drop it in the _plugins directory), so the "write-index-to-a-sqlite-file" part shouldn't be hard to build either.
Is FTS5 included in the SQLite browser builds? The SQLite amalgamation includes it by default.
I’ve always been a big fan of Jekyll because it’s so easy to use & so stable. This adds a lot of value to the ecosystem. I’ve built a bunch of faculty websites over the years where I needed a lot of structured, repetitive data (papers, honors/awards, etc.). It would have been so much easier to manage if I could have stored that data in a database instead of just flat files.
This is really cool. Whilst it's been fun mucking around with next.js etc (and arguably that's for a different purpose) for an out the box website Jekyll has proven itself time and again. Looking forwards to trying this out.
This is interesting. I suppose if you wanted to setup your Jekyll data relationally, this would make it simpler to pull a simple list of the combined data.
I wonder if DuckDB would make it easier to do this same thing and use the existing Jekyll data files?
In wake of all the WP craziness a few weeks back, I wonder how long before someone builds a "best of both worlds" CMS to rival WordPress. "You get the nice admin UI, but it generates a static site" type of thing.
Great project. I've long thought about creating a static-site transformer for wordpress sites via SQLLite-- in order to reduce costs and improve security.
The idea would be to feed the wordpress content into a sqlite DB and re-publish the entire site as a static site. Since wordpress comments have declined in usage, this should work well.
Publishing time would be a bit slower, but reads will be 100x faster and 10000x cheaper.
I have nothing to add other than Jekyll has been rock solid for me for, what, like a decade plus now?
I've ran swiftjectivec.com on it, and it's always been the perfect middle ground of taking care of the cruft I don't want to deal with, while allowing me to get my hands dirty and code when I want. Some of my favorite software ever.
I have some old jekyll websites that I only very infrequently need to update. Each time something in the ruby / gem / bundler / jekyll chain setup is broken, and with some weird errors it is stackoverflow search time. Very time consuming, highly annoying. I postponed my last typo correction, to first make up my mind on whether to port to astro that I use currently.
I actually switched my blog to use Zola (similar to Jekyll but packaged as a static binary instead of a Ruby gem) because I couldn't figure out how to build my site with Jekyll after a few years- it kept trying to compile C code?
Bear in mind this was 5 years ago and I had never used Ruby before, so probably a user error :)
I built endoflife.date with it, and it has been great. If I had to do it again, I might pick Mediawiki (or something similar) due to it being a community wiki more than a static site, but Jekyll hasn’t let us down yet.
Really appreciate you building and maintaining it these years! endoflife.date has been an amazing resource ever since you put it up. This has been a great open source success story that I share with people to explain why open source can be great. It's also a great example of the power of crowd-sourcing data. I originally added Ruby[1], Fedora[2], and Alma Linux[3] to endoflife.date and it's rewarded me with years of ability to reference. Prior to this I kept my own notes for the projects I cared most about, but keeping those up to date was a PITA. The best ideas seem obvious in hindsight, and endoflife.date definitely seems obvious :-D
Nice! Great to see innovation in the Jekyll space.
Quick FYI on Jekyll performance:
I noticed slow generation times on one of my sites and traced it to a plugin that was spawning a git process to get the last commit info for every page. I wrote a drop-in replacement called jekyll-last-commit[0] that uses the ruby libgit2 wrapper for improved performance. Details on its origins are in an old HN comment[1] if you are interested!
[0]: https://github.com/klandergren/jekyll-last-commit
[1]: https://news.ycombinator.com/item?id=34331663
The other last-modified-plugin is now archived, and no longer included in the default Jekyll, but the sitemap and seo plugin still seem to rely on the page.last_modified_at property, albeit undocumented (https://github.com/jekyll/jekyll/issues/9702), so might be worth getting the implementation switched upstream perhaps?
We show the last-modified date on endoflife.date products (https://endoflife.date/jekyll), but I'm not sure how we have the property set right now, since I don't remember including either plugin. Thanks for creating this, I have a few other sites where this will be quite helpful.
Although I don’t use Jekyll, loved this idea. I think using SQLite as the main data source would be a nice way to preserve content and play around with different static generators if they all had a plugin like this.
Also TIL about the baked data pattern, which I think is exactly what I needed for an upcoming project, so thanks for that. Though, I do align with one of the commenters here—this doesn’t seem like the same thing as the Baked data pattern in that Simon’s approach was about using a server rendered app with read-only data instead of generating a lot of static pages.
Nevertheless, this seems like a nice way for static generators to work with bit more complex data sources without dumbing them down to JSON/YAML.
I'm a little confused. The baked-data model is so you DON'T have to generate a thousand static pages. But this solution does exactly that.
Not complaining, mind you. My kid is trying to learn HTML/CSS/JS and wants to put together a read-only website with a database backend. I'll be pointing him this way as an ootion once he's far enough.
But it's still puzzling to link it to baked-data. Maybe I'm missing something.
> bundling a read-only copy of your data alongside the code for your application, as part of the same deployment
You can see https://github.com/captn3m0/northwind for example, which bundles the entire database alongside the code in the _db/northwind.db file. While Simon considers it primarily for dynamic apps, you have the ability to build PWAs and other interesting apps with the baked data pattern.
I'm building blr.today for example using this.
This is awesome, and would solve an actual problem I've been thinking through... if only I had selected Jekyll for my static site generator. Does anyone know of a similar solution for Hugo?
Technically I'm sure I could run a script to generate .md (or a format like .csv that Hugo can work with) from my database, but this seems like it might be easier for a database that updates frequently.
Nice to see this project here! I've been using it with the Steam API to publish a list of the games I've been playing on my personal website: https://yboulkaid.com/games
I found that separating data from content on a Jekyll site is a really powerful way to have anything from photo galleries, blog entries, book lists, easily changeable menus etc...
This is so cool, bookmarked it. It sort of get the impression that Jekyll is slowing turning away from being a boring static site generator to something way interesting.
what is wrong with "being a boring static site generator"?
Now it's easier to generate static sites that aren't so boring!
In this article I show you a project on how to develop a static blog in jekyll. https://chat-to.dev/post?id=296
In this one I show how to host it in various stupid ways https://fev.al/posts/blog-infra/
Though the idea of using a database to store the content could make things even better. Maybe sprinkle some redundant postgres?
When you went from Jekyll to Kubernetes, it was must have been like when Ted Kaczynski first learned about battery-operated timed fuses.
Enjoyed the read.
Looking forward to when you chuck the whole thing into the bin and run Wordpress on your linux home server, fronted by a free Cloudflare zero trust tunnel.
Is it possible to generate an SQLite DB from the site data and statically serve it, so sql.js can use it as a DB to provide something like search?
I'm mostly focusing on sqlite-as-the-data-source, but I can imagine something like this might be in scope for a project like lunr.js[1], which currently uses a large index served as JS/JSON. You can write a Jekyll plugin in a few lines of ruby code (with no gem management needed, just drop it in the _plugins directory), so the "write-index-to-a-sqlite-file" part shouldn't be hard to build either.
Is FTS5 included in the SQLite browser builds? The SQLite amalgamation includes it by default.
[1]: https://lunrjs.com/docs/index.html
I’ve always been a big fan of Jekyll because it’s so easy to use & so stable. This adds a lot of value to the ecosystem. I’ve built a bunch of faculty websites over the years where I needed a lot of structured, repetitive data (papers, honors/awards, etc.). It would have been so much easier to manage if I could have stored that data in a database instead of just flat files.
We use a similar “baked data” approach with Duck DB + a static site generator in evidence
https://github.com/evidence-dev/evidence
This is quite interesting. I always wanted to build something similar using SteamPipe (which can pretend to be sqlite/postgres) alongside Querybook.
THe craziest production with this approach that I've seen is the crt.sh website, which builds a full dynamic website with postgres and sql: https://github.com/crtsh/certwatch_db/blob/master/fnc/web_ap...
This is really cool. Whilst it's been fun mucking around with next.js etc (and arguably that's for a different purpose) for an out the box website Jekyll has proven itself time and again. Looking forwards to trying this out.
This is interesting. I suppose if you wanted to setup your Jekyll data relationally, this would make it simpler to pull a simple list of the combined data.
I wonder if DuckDB would make it easier to do this same thing and use the existing Jekyll data files?
In wake of all the WP craziness a few weeks back, I wonder how long before someone builds a "best of both worlds" CMS to rival WordPress. "You get the nice admin UI, but it generates a static site" type of thing.
Django-Distill I found to be pretty good for this. Use Django and the admin interface while being able to generate a static site.
There are a few of those, I think the term used to describe them is "headless CMS".
TinaCMS, fka Forestry is trying to do this. It's been a while since I've tried them out, but I love the idea.
https://tina.io/
Great project. I've long thought about creating a static-site transformer for wordpress sites via SQLLite-- in order to reduce costs and improve security.
The idea would be to feed the wordpress content into a sqlite DB and re-publish the entire site as a static site. Since wordpress comments have declined in usage, this should work well.
Publishing time would be a bit slower, but reads will be 100x faster and 10000x cheaper.
Wow, cool stuff!
Thanks!
I have nothing to add other than Jekyll has been rock solid for me for, what, like a decade plus now?
I've ran swiftjectivec.com on it, and it's always been the perfect middle ground of taking care of the cruft I don't want to deal with, while allowing me to get my hands dirty and code when I want. Some of my favorite software ever.
I have some old jekyll websites that I only very infrequently need to update. Each time something in the ruby / gem / bundler / jekyll chain setup is broken, and with some weird errors it is stackoverflow search time. Very time consuming, highly annoying. I postponed my last typo correction, to first make up my mind on whether to port to astro that I use currently.
> , to first make up my mind on whether to port to astro that I use currently.
Just sucks that eventually, that will happen with whatever you use after migrating too, the only difference is how long it takes.
I'm hoping NixOS or even just Nix for dev envs (or something similar) will help against this, so you end up with environments that just keep working.
I actually switched my blog to use Zola (similar to Jekyll but packaged as a static binary instead of a Ruby gem) because I couldn't figure out how to build my site with Jekyll after a few years- it kept trying to compile C code?
Bear in mind this was 5 years ago and I had never used Ruby before, so probably a user error :)
Glad it's been so stable for you!
> it kept trying to compile C code?
That‘s something that works incredibly well on Windows, better than I would have expected to.
The rubyinstaller.org people are shipping a fantastic installation. Under the hood it‘s msys2, but everything simply works out of the box.
I built endoflife.date with it, and it has been great. If I had to do it again, I might pick Mediawiki (or something similar) due to it being a community wiki more than a static site, but Jekyll hasn’t let us down yet.
Really appreciate you building and maintaining it these years! endoflife.date has been an amazing resource ever since you put it up. This has been a great open source success story that I share with people to explain why open source can be great. It's also a great example of the power of crowd-sourcing data. I originally added Ruby[1], Fedora[2], and Alma Linux[3] to endoflife.date and it's rewarded me with years of ability to reference. Prior to this I kept my own notes for the projects I cared most about, but keeping those up to date was a PITA. The best ideas seem obvious in hindsight, and endoflife.date definitely seems obvious :-D
[1] https://github.com/endoflife-date/endoflife.date/commit/7dae...
[2] https://github.com/endoflife-date/endoflife.date/commit/ab16...
[3] https://github.com/endoflife-date/endoflife.date/commit/12a7...
Thanks a ton!