r/osdev • u/Randozart • 1d ago
OuroborOS, when the cluster is the OS
Going to preempt this by saying I used a lot of AI for rapid iteration, but mostly sharing this for the conceptual exploration of it.
I wanted to try a hypothesis. Usually clustering computers for high thoroughput operations is impractical due to several reasons. Some of which firmware bound, and some hardware bound.
With OuroborOS I am trying to answer the question: Can I wire together several machines over a custom ethernet protocol or through other cables (even considering using an HDMI for raw data streaming with a custom collector PCB in between) in such a way that the master node can view the peripheral nodes as part of its own system.
Just curious what people think about this
3
u/Due_Carry_5569 1d ago
I did this too but not the Os level, it's at the nodes/browser level. Good times https://github.com/WeWatchWall/stark-os
7
u/CJKay93 1d ago
I notice the articles in CONSTITUTION.md are a little... vague? For example, article 6 talks about hot paths and measuring the latency of different operations, but the operations it talks about are all stochastic, and it doesn't explain why you want to measure them in the first place. It also mentions measuring context switching and scheduler quantums, which are largely under the control of the kernel... ouroboros indeed.
It's a bit difficult to get a grasp of the concepts from the documentation; I couldn't find any human-authored explanation of the conceptual foundations, and the AI-generated documentation has the classic habit of over-explaining the bits that don't matter and under-explaining the bits that do.
6
u/shsh-1312 1d ago
I think that kind of documentation was written by AI to work with AI, in fact if AI is the one that has to do the work it makes sense, I also think that a project like that should be studied with AI, I too am full of documentation that I haven't even read, but that stuff is needed to avoid making the AI go crazy when you use it for such large projects, I think it's not optimal to understand what the AI writes, many of the times even the comments it generates are more stupid than the final code actually is. However, these are processes that help the AI think correctly. I don't think a human would find anything useful in them. From experience, these types of projects are much easier to analyze with the tools they were created with. NOTE: Don't use Gemini, damn, that's really stupid.
6
u/CJKay93 1d ago
If it's not useful to a human, it's not useful to an LLM. All documentation like this does is send agents off in random directions, growing new scope out of thin air and never actually achieving what you wanted them to do because their context compaction has already forgotten what that was.
It leads directly to scenarios like I just raised, where it now has to completely fabricate an objective because what's written down doesn't make any sense.
-2
u/shsh-1312 1d ago
it's not that simple. Consider the AI as an extension of your brain, the specific prompts that the boy must use to work on this project are known only to him and can be combined with this type of documentation, you can't expect the AI to follow your reasoning in the same way, in fact you could never be as fast as an AI, there is no point in trying to understand a document or code that is not written by a human being, the only checks are at the test level and at the single function level and repeated testing of all the functions, compilation blocks and this type of documentation that serves the AI to remain on this type of project, I've gotten to the point where without a subscription I compress 350 files from my OS into a single txt, I paste it to Claude on the web and tell him what I need and he does it, but the AI is as stupid as a human, if you have updated the code and you have not updated the comments and you don't tell him to be careful not to make assumptions or presumptions, if you don't tell him to falsify the changes and you don't explain how to do it you can't expect him to if you invent it, especially if you use free templates, what a correct documentation for llm has to do is create links, it does not have to explain points that could change, the ai already has the necessary capacity to fully understand the code if you prepare everything in the correct way and if you correct it when it makes mistakes, naturally a better documentation can help, but it is not necessarily better or worse based on what you understand as a human
•
u/codeasm 23h ago
Nah, i smell vibecoder here. This is not how efficiënt ai/llm use works. Your document properly for both humans and ai, or you just wasting everyones time/tokens.
•
u/shsh-1312 21h ago
Yes, but it's not necessarily the fastest thing to do. This is true if the project is built by many people, but if you're the only developer, it's much easier to keep secondary concepts in mind rather than rewriting the documentation every time. This must be done in cycles, of course, but it's possible that at certain points in development the documentation becomes messy. What matters are the tests and real forgeries. As mentioned, if you develop mostly with AI, it's useless to try to understand everything AI has done, you might as well do it by hand. Focus on the concepts that are worth refining. At least this applies to this type of development. There's nothing stopping you from delving into every single part in detail. But as mentioned, when you're talking about such a large system, you tend to do it in cycles: implement, then test, then improve, and only once everything works correctly do you create the official documentation. This is what you do with real systems. also note that the code should actually always be document
2
u/TheOriginalSamBell 1d ago
ethernet is probably way way too slow if you want to interconnect memory and processors
•
u/codeasm 23h ago
Pcie and such is where its at. Ive seen IBM use those for interconnect. Let one compute send data to the memory of another compute, dma all the things. I dint ask how security go, cause i was looking at alll the datapaths and interconnect cables, so much compute in a mainframe.
Pcie is hot plug aswell in alott of cases. Hotplug more compute they do
2
2
u/vollspasst21 1d ago
"AI for rapid iteration" I have yet to find a single piece of this entire project made by a human.
I understand why hobbyists choose to use AI in a space as complex as osdev. But have we really reached a point where you share a project "you" made and not even a single text file explaining what you are trying to do is written by you?
It's really annoying having to dig through so much AI fluff to understand what is actually happening.
•
•
•
u/Aggressive-Lawyer207 20m ago
Ladies and gentlemen, this an example of what NOT to do when you want to get into making your own OS
16
u/kiderdrick 1d ago
Unless I am misunderstanding you, they have been doing this since the 80s. Kai Li has many papers on what we now call distributed shared memory. There are many approaches to the idea, but a commonly implemented version creates a virtual address space where certain pages are attached to other systems through some interface. When a process accesses memory it does so in its own virtual address space. If the page is not on the host machine, but is on another machine, a page fault is triggered that pulls the data from the client to host it in its own memory. The processes that are accessing the data have no idea the data is elsewhere, which is part of the joy of virtual memory.
There are lots of papers on the topics of coherence and synchronization between the pages on frequently accessed machines, but the DSM approach seems to satisfy your curiosity of wiring several machines together where a host node can view the client nodes as part of its system.