It looks cool, but I would be hesitant to leave IaC behind. DevOps/sysadmin kind of people are infamous for (generally) not liking GUIs for a reason. We like being able to grep through a repo, revert to older versions of the code, and deploy new infra all through the CLI.
It might be adopted by people who already do ClickOps or aren't comfortable with the CLI, but I'm not sure about the rest of us. Looking forward to seeing how it progresses.
Skepticism here is warranted - the history of things that look like this in our space isn't fantastic. We've put a lot of engineering into trying to create something that is a power tool, and flexible enough to solve hard real world problems. It's early, and we've got lots of work to do, but that's the goal.
I heard in a Podcast that they are inspired by game development workflow (e.g. Unity) where multiple teams can work together in a unified way. This reduces "niche" power like CLI commands that only some of sysadmins can wield.
Personally, as an infrastructure engineer, I still prefer CLI and everything as (text) code. But I'm open to change if this "second wave" could take off.
As someone who works on games professionally, the first thing people do is turn on the text support for unity assets. Unreal still uses binary assets and it's a giant pain in the ass.
The problem isn't that everything should be text, it's that there are no tools out there that work with other content types. Semantic diff tools would be incredible honestly, and text is a poor representation of code. It's just the best we have right now
I tend to romanticize CLI as a universal and think of GUIs as niche, so it struck me odd to see CLIs referred to that way. That's not a character judgement. I'm actually curious what you mean by niche. Would you mind elaborating?
Infrastructure components should still be written in code but composing multiple components (network, infra, database) is often a Byzantine maze of different tooling/language choices (terraform/pulumi/crossplane) and multiple pull requests.
If you shift to an application developers perspective, and zoom out a level most medium/large org still require interaction with multiple teams to spin up an application. Testing and deploying this stack end to end is expensive and time consuming.
I’m not sure what the optimal solution looks like, but certainly open to tools that force us to think more about building interfaces around infra components and wire it together without navigating PRs in a dozen different repos. PR should be the process to create a new version of a component, but not an API for users requesting an instance of that component.
Hi holoway, I've been following your project for a couple of months now. How do you compare yourselves to upbound/crossplane?
IME crossplane has been a "much better terraform" and also borrows from some of tf's open source provider code. Seems to be one of the best IaC pattern for k8s centric shops.
Pros:
- adoption of k8s core engine, state mgmt
- model everything as a CRD, consistent definition pattern both infra and app
- open source
- great UI when layered with argoCD
- declarative
Cons:
- steep abstraction learning curve (for me anyway)
- docs lacked key context for newbs (also getting way better, big efforts here)
I think you hit the nail on the head with crossplane being a 'much better terraform' by design. Our goal isn't so much a better IaC / Declarative infrastructure tool - it's a better overall workflow for doing collaborative DevOps work. We think that by having an active model of your component, and tracking the resources along side, we can fix the feedback loops in a way that things like crossplane, terraform, or pulumi really can't.
Of course today it's early - so you have to look at what we're building as a foundation for the future. But it's a solid foundation to build on!
Wait, I thought k8s and open shift had this entire service catalog concept that was basically sort of a CRD->IaaS bridge.
Did that die? Is that not even what crossplane is? Once again, I'm 6 clicks deep and havent seen a single actionable example. I wish I could get inside some people's heads and wonder wtf theyre thinking when marketing this stuff.
Edit. Wow OSB and Service Catalog are dead. Is there a obituary?
> Once again, I'm 6 clicks deep and havent seen a single actionable example
Based on the terse reply to my question, it seems they really want you to create an account and go through their onboarding tutorial after agreeing to an absolute telephone book sized ToS. Hell, it's possible I'm violating one of the paragraphs by even talking about the process. I appreciate that's likely not what you had in mind, but it seems to be what they have in mind
I read their website, their Github project page, and still have no idea what it is (but I know how to install it!). I started to read their blog post but it's too long and while they claim this tool is useful for DevOps, I couldn't find any info on how it accomplishes that.
The website seems to have more information https://www.systeminit.com/ the video is enlighting. I'm unconvinced by their product though, I'm not sure that this is a good approach.
I was interested in the concept but after looking at the source code and the architecture I think I agree with the "not sure that this is a good approach" conclusion.
Used identical tools before. It's just another vendor lock in monoculture. Doesn't matter that it's open source. The design is such that it's all or nothing. Same as Terraform.
One side effect of our design is that, once we bring more of the bi-directional capabilities back into the product - we can track the real-world resources as they change, and update the model on your behalf. So you can keep using (or start using) whatever tool you like to make the changes, and SI will do its best to help you rationalize that. We've had this working in earlier versions, and will again in the not too distant future.
The interface looks sexy. The binding to paid services instead of binding to APIs of open source systems makes me just sigh and go on with my life without System Initiative.
Let me preface this by saying - if this works well enough to give devs confidence in the tool, this could be revolutionary. It would combine implementation, documentation, visuals and deployment into a single view simple enough for everyday devs to grok.
I couldn't find any documentation - if there is, that answers these questions, please link me to them.
1. How does authentication work? Is it handled by the tool?
2. Can it support non-AWS resources, such as Grafana?
3. How does version control work? Can you review changes, roll back, etc?
4. In the demo on the main site, the security group was not attached to any VPC, what's going on there?
5. Does this remain maintainable when dealing with very large configurations, for example, a multi-account AWS setup with tens of thousands of resources. If so, what mechanisms are available in this tool to facilitate this maintainability?
It's very early - it doesn't do much yet (but what it does do is compelling!). There isn't a ton of documentation, but we have put a lot of real user research into things.
1. Authentication works through Auth0. We're building towards multiple deployment models, where your account works across all of them. That said, it's all open source, so if Auth0 doesn't work for someone, we're happy to make it pluggable.
2. You can model anything you like. Under the hood it's a hypergraph of Typescript functions - so you would model Grafana, and then make calls to its API when actions are needed.
3. It's built in to the model via change-sets. As you do change the model, we show you whats changed, and what actions we would take. The design is heading toward letting you have comprehensive reviews based on which portions of the entire model are impacted. Roll-backs aren't really a thing in infrastructure land, but you can see old versions of the model and decide you want that to be the current one.
4. We model the upstream 1:1 - so that configuration uses the default VPC in the AWS account (which has likely been deleted if you use Terraform, for example.)
5. It's too early to have very large configurations yet. But there are techniques we can borrow from other domains - nesting, for example, or layers. We're working on the fundamentals first, and then we will deal with scale.
What were the decisions supporting the project's use of Buck2? Considering your team is specializing in the domain of infra automation, compile times of all of SI's Rust must be a hot topic among the team.
We've got a fairly large monorepo, with code primarily in Rust, Typescript and PgSQL. We wanted something that would scale as we grew into things like remote execution and build farms, but that would let us be pragmatic in the meantime. Buck2 fit the bill at the time we needed to solve the problem. We also needed it to understand cross language dependencies.
One of the great things about it is that our CI system uses BXL to automatically generate pipelines on the fly from impacted code, taking into account dependencies. So things generally move as quickly as possible through the system.
It's been real work to adopt, but the upside for us has been worth it.
the ability to not have sandboxed local builds meant that it was easy for us to write the starlark/python code we needed to get things working in a rough way quickly, while correctness can come later.
Was this something you had planned to announce prior to the whole Terraform kerfuffle and it is just unfortunate timing, or was this precipitated by that event in any way?
We've been planning to open source it this way for the better part of a year. It just so happened that we were ready to open it up right after the Hashicorp folks made their decision. No intentional timing.
I got through the first half of the tutorial. I have a busy week but I was excited to see this. It’s right in line with tool I’ve been trying to cobble with distracted effort for nearly a decade. I am so excited to continue to use this tool. Y’all nailed it.
No, I say that when the account appears to have been created for that purpose, or if there's reason to believe that the same user is creating multiple accounts to abuse HN. I might have just gotten this wrong in this case. Thanks!
If you really believe that then maybe you should stop commenting on HN because I'm not looking forward to the time when you decide to want to take your marbles home and you too start spamming every submission because you don't get your way.
We try not to delete entire account histories because that would gut the threads the account had participated in. However, we care about protecting individual users and take care of privacy requests every day, so if we can help, please email hn@ycombinator.com. We don't want anyone to get in trouble from anything they posted to HN. More here <https://qht.co/item?id=23623799>.
and when you architect managing user privacy as a drawn-out process that won't even begin until another human responds in a completely manual engagement, i can immediately see the motivation of the above poster wanting administration to provide an answer without the response time being calculated in business days.
completely fair game to shortcut an outdated hostile process with your own hostility, not really an unexpected result (e.g.: web scraping VS using an inaccessible API). it's an unacceptable approach in a modern context.
I once asked to change my username. This was done within hours. I once asked for my account to be disabled, if I recall correctly this was done within a day. I asked for it to be re-enabled a year later, and this was done within hours.
All of this took a single email every time.
I don't know what "Hacker News does not allow users to delete comments"-person wants or what they did or didn't ask at hn@, but it's completely valid to NOT nuke entire threads just because one person has a hissy fit. Reading old Reddit threads (and sometimes even recent Reddit threads) can be a real pain because of this. If they want their content anonymized or their account disabled they can do that easily, if they want some specific post altered or removed then I'm sure Dan will be accommodating.
If that's an accurate representation, then I'd see your point.
There are a few circumstances which might suggest hesitancy, e.g., has your account been compromised and the person requesting deletion isn't the original owner? But barring that, if you're requesting deleting your content, and you're clearly exercised over it, HN should delete your content.
An alternative might be to rename the account to something that's not traceable to you, though how effective that is will depend on what you've included in earlier comments or posts, and whether or not that's archived somewhere. (I don't know what or if you have done this, I do note that your account dates to 2011.)
> We're always happy to delete or redact identifying information. We don't want anyone to get in trouble from anything they posted to HN. We don't do it by allowing wholesale deletion of account histories, because that would gut the threads the account had participated in, which would be unfair to any commenters who replied, as well as to readers and to the community.
> That doesn't mean we don't care about individual users' needs for protection—we care a lot about it, and help people with these requests every day. We just have to do so with sharper tools, and we have a big bag of tricks for taking care of these things. They include renaming accounts, retroactively assigning comments to throwaway accounts, deleting specific posts (especially if they don't have replies), redacting specific info from posts, and more.
---
Neither GPDR nor CCPA require the deletion of the account or content contributed that does not fall under personally identifying information.
If there are comments that contain PII, that should be sent to HN. Make sure that it is clear that these are PII that contains information that is directly linking to a real person.
Though as noted, wholesale account and comment deletion isn't required and is disruptive.
I'm aware that accounts get renamed. I've run across a few instances, most recently through my front-page activity analysis. IIRC there are some glitches, possibly with Algolia still referring to the old name, or the old name showing up othewise in some archives.
I also strongly suspect there's an independent (and immutable, or at least more durable) userID value that's not the semantic name, which is kept internally to HN, and isn't publicly exposed (at least not that I've found).
Well, do you allow account deletions then? It should be supported under GDPR. We discuss this about other tech companies all the time here on HN. I never realized this is not possible on HN.
GDPR does not give a right of account deletions; it gives a right to delete personal data, and even that is caveated, and comments you leave on a site like this as not really "personal data" in the way that's intended with the GDPR.
It might be adopted by people who already do ClickOps or aren't comfortable with the CLI, but I'm not sure about the rest of us. Looking forward to seeing how it progresses.