While I was searching on how to switch to graphics mode so I can support Arabic text in the console, I ran into the Linux Framebuffer project. It makes their console supports graphics and unicode - and evening basic windowing and events. I looked a little into the source, basically they implemented tens of times, each for a specific VGA vendor or so. The important thing is that they really have a biiig work to support text on that "console" - A whole folder. While there were hundreds of files in that project, I saw only two or three. Each one not less thant 3000 lines. Especially the fbcon.c :S. I checked what stuff they call in the kernel, they at last include 20 header files of linux kernel, so porting is not an option. They also call semaphores and other stuff. So I think it is still too early for this kind of work for this OS - at least to be such integral and perfect. Maybe basic support for arabic character will be much simpler - I hope!
It is - as I seen in the linux kernel - a long way for us, but we will never arrive there if we didn't move - even if a cm per year. By saying "oh, it too long way, I won't do it" we are going no where. I believe in that integration over a very tiny development produces a big piece of work, but if there is no tiny development -"continuously" by the book - it won't be integrable and nothing would be produced.
For now I am still working on the low-level allocator. I think we're past the phase just-do-something-that-just-works. Making such thing will make it harder to optimize and maintain. So I am taking the very slow careful way. Maybe even at times leaving it and returning to older parts of other modules. At this time I am seeking code consistency and maintainability. Making something that just works and returning to make it better later means that you *will* rewrite it, it is a fact. The only problem here is that we didn't make an OS before we don't know how maintainability needs at this level. We yet have to keep going till we know, which is bad thing. The only thing that can make that better is to go for new modules and return to older code, and so on for every new aspect of the system to insure integrity and consistency and expected behaiuor is really what is expected.
Do you think I should move all this OS stuff to a seperate blog ? I am thinking in it.
Saturday, July 22, 2006
Friday, July 21, 2006
ArabOS Road map
Enable A20 line ( to address the full 4 GB): done by grub
Switch to protected mode and 32 mode: done by grub
VGA Memory text mode management: done
GDT,IDT,PIT,IRQ Remapping: done
Paging: done ;)
Memory management: under development
multitasking: under development
-security,users,sharedpages,priority,copy-on-write
IDE drive: future work
File System
Porting LIBC and Bash and GCC: future work
Executables loader
Inter Process Communication IPC
VM disk swapping
Device support
- DMA
- PnP
- Device drivers
--- MicroKernel
Switch to protected mode and 32 mode: done by grub
VGA Memory text mode management: done
GDT,IDT,PIT,IRQ Remapping: done
Paging: done ;)
Memory management: under development
multitasking: under development
-security,users,sharedpages,priority,copy-on-write
IDE drive: future work
File System
Porting LIBC and Bash and GCC: future work
Executables loader
Inter Process Communication IPC
VM disk swapping
Device support
- DMA
- PnP
- Device drivers
--- MicroKernel
Thursday, July 20, 2006
ArabOS design decisions 1
Design decisoins is something that SW developers do when they program. But OS design decisions have a big influence that one decision can remove and add thousands or even millions of lines of code. Anyway this article is mainly about the design decisions of ArabOS and what I did choose and a rationale.
The first decision is to use a premade bootstrap loader, or a custom one. I read that for new OSs it is better to use a premade loader so one can focus on the development of the system rather than reinventing the wheel. Although I have made the bootstrap loader a long time ago, I decided to use GRUB (Grand Unified Boot Loader). It makes so many things for you. It supports Multiboot, loading from a CD or a flobby. It initialyzes 32-bit protected mode for you. It also queries the BIOS for the reserved memory areas.
The second decision is to use GCC or MSVC. Well, I didn't choose; I tried. Some stuff didn't work with MSVC, especially the multiboot support thing, and special functionality only provided by the linker script of GCC(LD). So I had to move to GCC, since having problems from that early stage, what whould happen after a long time ? (Also GCC have support for OS development)
The third one is to use NASM or GAS. After trying, especially I am not professional in GAS, I had to do stuff in NASM (long jmp didn't compile with GAS, I don't know why). I am using most of the asm files as NASM, but I am intending to move to GAS later.
Next is to develop under Linux or under Cygwin. Actually, Cygwin is PAIN. I can't use ELF file format which prevented me from linking my kernel to run in the upper 2 GB VM (Thanks to GRUB which didn't understand PE file format). But I used Cygwin because I used VMWare. Now using Bochs which run under Linux, I can move to Linux anytime. But I'd rather after a while.
After that paging VS segementation; Upper half VS lower half; has predefined answers. Paging and Upperhalf. Search for them.
C VS C++. Well, C++ in an un-hosted environment is a pain. I have to make the runtime support my self, not even the whole thing. So using C99 is actually fine and comfortable as long as I don't need namespaces, templates, and non-singleton classes.
VGA VS Text-Mode. Text mode. There is tons of VGA drivers out there, XServer+KDE will do it all.
Monolithic VS Microkernel VS Exokernel. In the current phase of development, I am going for monolithic cause of simplicity. Later I can switch to Micorokernel. I don't think Exokernel is an option.
Single or mutiple platforms. Not now, it's too early, so I'd go for single. Same for multiple CPU support.
The first decision is to use a premade bootstrap loader, or a custom one. I read that for new OSs it is better to use a premade loader so one can focus on the development of the system rather than reinventing the wheel. Although I have made the bootstrap loader a long time ago, I decided to use GRUB (Grand Unified Boot Loader). It makes so many things for you. It supports Multiboot, loading from a CD or a flobby. It initialyzes 32-bit protected mode for you. It also queries the BIOS for the reserved memory areas.
The second decision is to use GCC or MSVC. Well, I didn't choose; I tried. Some stuff didn't work with MSVC, especially the multiboot support thing, and special functionality only provided by the linker script of GCC(LD). So I had to move to GCC, since having problems from that early stage, what whould happen after a long time ? (Also GCC have support for OS development)
The third one is to use NASM or GAS. After trying, especially I am not professional in GAS, I had to do stuff in NASM (long jmp didn't compile with GAS, I don't know why). I am using most of the asm files as NASM, but I am intending to move to GAS later.
Next is to develop under Linux or under Cygwin. Actually, Cygwin is PAIN. I can't use ELF file format which prevented me from linking my kernel to run in the upper 2 GB VM (Thanks to GRUB which didn't understand PE file format). But I used Cygwin because I used VMWare. Now using Bochs which run under Linux, I can move to Linux anytime. But I'd rather after a while.
After that paging VS segementation; Upper half VS lower half; has predefined answers. Paging and Upperhalf. Search for them.
C VS C++. Well, C++ in an un-hosted environment is a pain. I have to make the runtime support my self, not even the whole thing. So using C99 is actually fine and comfortable as long as I don't need namespaces, templates, and non-singleton classes.
VGA VS Text-Mode. Text mode. There is tons of VGA drivers out there, XServer+KDE will do it all.
Monolithic VS Microkernel VS Exokernel. In the current phase of development, I am going for monolithic cause of simplicity. Later I can switch to Micorokernel. I don't think Exokernel is an option.
Single or mutiple platforms. Not now, it's too early, so I'd go for single. Same for multiple CPU support.
Monday, July 17, 2006
C# is subway, C++ is automobile
I've been thinking of an analogy between C# and C++ to the real world. Suddenly It popped into my head when I was in the subway. I was thinking of comparing the user of the subway to the driver of a car but it didn't seem fair so I decided to compare the subway driver to the car driver.
As a subway driver you don't have to steer, nor watch for intersections. In C# you don't have to make hard design decision, not to watch for platform support. But as a car driver you have to steer, change gears, and watch your sides even not in an intersection. Likewise in C++, you sometimes have to decide whether to sacrifice one thing for the other, usually memory VS time, or flexibility VS simplicity. You also have to watch different platforms Endianess, int size, and such stuff.
In subway you have predefined stations, and the path is optimized to fit for those, it would be costly to head somewhere that is not near a station. In C# you have predefined intentions in the language and the framework design, you would suffer if you decided to do something that wasn't intended to be done. For example, C# is business-oriented; its classes are best fit for common business tasks; would you consider doing a NASA space shuttle software using it ? In a car, you can virtually go anywhere, provided you have the right directions- and enough gas. Not always the paths will be perfect but some paths can be even shorter than the subway path, If you were an experienced driver or you have a good GPS map. In C++, you can do anything, provided you have the theoretical knowledge- and a linker that can link to ASM in case of system programming. You can do stuff in C++ by nature needs efficiency. Try to write a memory manager (in the system programming context) in C#, it is possible but it would take more time than to do it in C++ WITH the same efficiency.
Driving a subway is boring by nature, but it can be fun if you have a big speed. In C#, you do nothing special, every thing is ready for you. It would be fine in case you are making an enterprise-scale project. In a car, driving is boring and hell for people who can't drive well, they will keep banging in every turn, but if you know the tricks of it, and fair knowledge of the motor and its capabilities, you can do magic- see NFS: Underground 2. In C++, it is hard and ugly code if you don't have the essence of mastering it. If you do, you Will enjoy it, and can do software you never imagined it is possible- read for an example about loop-unrolling.
You would absolutely use subway if you are gonna a predefined place and you don't want to make a lot of effort. But you would use the car for a custom path that is not in the subway's map. Same in C# and C++, you don't have to use only one for all of the time. Just use what is suitable for what you wanna do. And remember, knowing only how to use the subway or the car, is not good enough, cuz at some point you will need the other.
Finally, some points here may have personal influence because I like C++. I admit I hate using C++ for some tasks, but I like its power and flexibility regardless of its complexity.
If you have another opinion or seeing something missing, you are welcome to comment.
As a subway driver you don't have to steer, nor watch for intersections. In C# you don't have to make hard design decision, not to watch for platform support. But as a car driver you have to steer, change gears, and watch your sides even not in an intersection. Likewise in C++, you sometimes have to decide whether to sacrifice one thing for the other, usually memory VS time, or flexibility VS simplicity. You also have to watch different platforms Endianess, int size, and such stuff.
In subway you have predefined stations, and the path is optimized to fit for those, it would be costly to head somewhere that is not near a station. In C# you have predefined intentions in the language and the framework design, you would suffer if you decided to do something that wasn't intended to be done. For example, C# is business-oriented; its classes are best fit for common business tasks; would you consider doing a NASA space shuttle software using it ? In a car, you can virtually go anywhere, provided you have the right directions- and enough gas. Not always the paths will be perfect but some paths can be even shorter than the subway path, If you were an experienced driver or you have a good GPS map. In C++, you can do anything, provided you have the theoretical knowledge- and a linker that can link to ASM in case of system programming. You can do stuff in C++ by nature needs efficiency. Try to write a memory manager (in the system programming context) in C#, it is possible but it would take more time than to do it in C++ WITH the same efficiency.
Driving a subway is boring by nature, but it can be fun if you have a big speed. In C#, you do nothing special, every thing is ready for you. It would be fine in case you are making an enterprise-scale project. In a car, driving is boring and hell for people who can't drive well, they will keep banging in every turn, but if you know the tricks of it, and fair knowledge of the motor and its capabilities, you can do magic- see NFS: Underground 2. In C++, it is hard and ugly code if you don't have the essence of mastering it. If you do, you Will enjoy it, and can do software you never imagined it is possible- read for an example about loop-unrolling.
You would absolutely use subway if you are gonna a predefined place and you don't want to make a lot of effort. But you would use the car for a custom path that is not in the subway's map. Same in C# and C++, you don't have to use only one for all of the time. Just use what is suitable for what you wanna do. And remember, knowing only how to use the subway or the car, is not good enough, cuz at some point you will need the other.
Finally, some points here may have personal influence because I like C++. I admit I hate using C++ for some tasks, but I like its power and flexibility regardless of its complexity.
If you have another opinion or seeing something missing, you are welcome to comment.
Friday, July 14, 2006
ArabOS news 1
Hiiii again :D
Finally I have managed to fix some annoying bugs about the cursor update and the programmable interval timer. I also have added a starter basic keyboard support. It seems clear now how to support DMA, and support devices. I am now being busy in reading about paging and virtual memory. Process management needs memory management. And virtual memory needs a file system. So I will not support VM now, just physical memory. You can visit http://del.icio.us/meemoxp for links about all that. I add each link I see interesting to it.
A quick summary about the OS state:
PIT: working fine.
Terminal: very basic but working fine.
KB: very basic and working fine.
I mean by very basic that the H/W stuff is working and all the remaining features are software stuff.
And by the way, another killer bug was also for using && instead of & !!!
The most of the past weeks was mainly for cleaning the code, setting constants instead of magic values, commenting on code, editing the makefile and well-structuring it, modifying the linker script, and fixing some bugs.
The main problem that you can't count on the code working well for being bug-less, and that you can't count that there is no better way of doing things. You also can't count that you know all features provided by each of the chipsets on the mother board, like APIC etc. You also can't count that you can catch and solve all the exceptions (especially hardware ones) because some may only occur at very rare occasions. Some exceptions once discovered might make you change more than 3 files and change a big structure of the system, especially at this early stage of development.
I am now going for monolithic kernel, not microkernel, and going for stack-swapping for process scheduling, not TSS.
Finally I have managed to fix some annoying bugs about the cursor update and the programmable interval timer. I also have added a starter basic keyboard support. It seems clear now how to support DMA, and support devices. I am now being busy in reading about paging and virtual memory. Process management needs memory management. And virtual memory needs a file system. So I will not support VM now, just physical memory. You can visit http://del.icio.us/meemoxp for links about all that. I add each link I see interesting to it.
A quick summary about the OS state:
PIT: working fine.
Terminal: very basic but working fine.
KB: very basic and working fine.
I mean by very basic that the H/W stuff is working and all the remaining features are software stuff.
And by the way, another killer bug was also for using && instead of & !!!
The most of the past weeks was mainly for cleaning the code, setting constants instead of magic values, commenting on code, editing the makefile and well-structuring it, modifying the linker script, and fixing some bugs.
The main problem that you can't count on the code working well for being bug-less, and that you can't count that there is no better way of doing things. You also can't count that you know all features provided by each of the chipsets on the mother board, like APIC etc. You also can't count that you can catch and solve all the exceptions (especially hardware ones) because some may only occur at very rare occasions. Some exceptions once discovered might make you change more than 3 files and change a big structure of the system, especially at this early stage of development.
I am now going for monolithic kernel, not microkernel, and going for stack-swapping for process scheduling, not TSS.
Thursday, July 6, 2006
Japanese section 1
I have attended my first section yesterday, and the instructor was nice :D.
We have taken 15 characters and about 6 words. The characters are 5 vowels, 5 k, and 5 s:
あa
えe
いi
うu
お0
かka
きki
こko
く ku
けke
さsa
せse
そso
しshi
すsu
The words is:
うえ
こえ
あし
and some other words, kefaya 3leekom keda :D
We have taken 15 characters and about 6 words. The characters are 5 vowels, 5 k, and 5 s:
あa
えe
いi
うu
お0
かka
きki
こko
く ku
けke
さsa
せse
そso
しshi
すsu
The words is:
うえ
こえ
あし
and some other words, kefaya 3leekom keda :D
Wednesday, July 5, 2006
Japanese How to program, er, how to learn
I finally got the opportunity to join a Japanese language course with A.Mohamadeen. I am really happy as I like that language very much. I have been trying to self-study it a year ago. It is easy, but self-studying it will take a really long time. I wonder if blogger.com supports Japanese, as I think I will isA occasionaly blog using it(for practicing). I am so exited that I will attend my first section today :D I hope the instructor is nice :).
Monday, July 3, 2006
The bright side(Masr lessa be7'eer)
I have seen a good commercial on MBC, showing the dark side then its bright side. Like, failure then ambition, dispute then democracy, I don't remember them all. I have really been sick from the pessimistic environment around us. I want to look to the good side a little. This article is about some of the good sides of the current situation of Egypt.
The education in our schools and universities at the average level dosn't seem to be quite comparable to other educational systems. Some students graduate not knowing even how to write correct Arabic, not mentioning English; how would there level be in other academic subjects they studied then! The bright side is that some students refuse to be graduated-ignorants, they study and study by them selves, maybe even reaching higher levels than other educational systems. There is also students who try to help younger students instead of leaving them by their own.
As proven, higher density of population causes aggressive manners. That's why you always see a man yelling from his car at other cars' drivers for the least cause, if not physically hitting them with the nearest piece of metal. The bright side is that if you fall out of sickness suddenly in the street you will find thousand hands helping you out. Maybe one of these hands is the hand of the person you've been yelling at. El masryeen ged3an ya3ny.
If you are for example making an OS, you will find people saying “Will you do something new, it is useless, you can't beat the XYZ's OS, you better learn something to help you in your career, etc,..”. You will hear a lot of this discouraging statement for making an RDBMS too. People who say that in my opinion discourage development of our country. Because this statements are applicable to any industry in the world from creating a pen to creating a satellite. El mafrood en e7na nas3a wel tawfi2 3ala rabena. The bright side is, as you find these people, there is other poeple who encourage development and ready to support it at any cost. I have encountered appoximately the same number of both sides. I have been recieving several emails offering to help in the OS (Thank you all Very much, your offer are greatly appreciated wa gazakom Allah 7'ayran).
Banks theifs, stole billions from this country. And bribees who take illegal extra money for making you a civil service. The bright side is that I have seen people in responsibilty position refusing to take their right so other people can take their right in full; because of lack in resources. I also have seen people who can make you a service for free to help this country(For my arabic teacher in my pre-high school who repeated the lessons that I didn't understand for me after school for free, not even knowing my name).
I have a read an article describing the environment of Al-Andalus. Arabs and Muslims was the top of science. And non-arabs was racing to learn Arabic they even couldn't write in Latin correctly. El Basra had 11 universities people come to it from every where. I think this is the reversed image of what's happening now. If they could reverse the picture so can we.As bad things exist, good things also exists, how ever big the bad side seems. As bad things get bigger, the good side gets bigger as equal, even if we don't see it.
If you see the bright side in something please send it for all of us to feel optimistic :).
Saturday, July 1, 2006
ArabOS v 0.003 - IDT : ISR and IRQ
It's been a long time since I wrote in this blog. I needed to have a rest after the projects.
Anyway, I have finally managed to handle interrupts. Loading the IDT (Interrupt descriptor table). Remapping of the IRQs to avoid conflicts with the reserved interrupts.
It was too tiring since this is actually the first time I need serious debugging. I had a strange problem the OS re-executes when any interrupt happens. It took me two months to know where the bug was, I had no way of debugging cleanly. Finally I sent to alt.os.development news group to discover that I just used && where I should have used & so that the address of the handler was the entry point of the OS; whay a coincidence! I couldn't have discovered this if they didn't give me a hint on how to debug. I used VMWare to test the OS, and I used messages to see where the execution went. They told me use the Bochs virtual machine because it has a debugger that lets you see the IDT, set break points and step. It helped me very much. But I was freaked out from myself. Staying awake two days, in 3:30 AM, having a screen with a lots of green data moving fast (seemed like The Matrix), reading assembly code, and counting how much bytes that instruction takes, and what's on the stack and what's bits are set in the machine Flags! I felt like a real nerd :(.
Anyway, the next step in the OS isA is to set the PIT (programmable interrupt timer) for the process manager, enable paging for the memory manager, handling keyboard and tty's, and start designing the architecture for the OS. I am leaving the File system for the end; the Dessert :D. After loading bash successfully, detecting devices comes after, then virtual memory.
All that will come in details in it's time. That was only the rough plan.
Anyway, I have finally managed to handle interrupts. Loading the IDT (Interrupt descriptor table). Remapping of the IRQs to avoid conflicts with the reserved interrupts.
It was too tiring since this is actually the first time I need serious debugging. I had a strange problem the OS re-executes when any interrupt happens. It took me two months to know where the bug was, I had no way of debugging cleanly. Finally I sent to alt.os.development news group to discover that I just used && where I should have used & so that the address of the handler was the entry point of the OS; whay a coincidence! I couldn't have discovered this if they didn't give me a hint on how to debug. I used VMWare to test the OS, and I used messages to see where the execution went. They told me use the Bochs virtual machine because it has a debugger that lets you see the IDT, set break points and step. It helped me very much. But I was freaked out from myself. Staying awake two days, in 3:30 AM, having a screen with a lots of green data moving fast (seemed like The Matrix), reading assembly code, and counting how much bytes that instruction takes, and what's on the stack and what's bits are set in the machine Flags! I felt like a real nerd :(.
Anyway, the next step in the OS isA is to set the PIT (programmable interrupt timer) for the process manager, enable paging for the memory manager, handling keyboard and tty's, and start designing the architecture for the OS. I am leaving the File system for the end; the Dessert :D. After loading bash successfully, detecting devices comes after, then virtual memory.
All that will come in details in it's time. That was only the rough plan.
Subscribe to:
Posts (Atom)