- Site menu:
- Latest Discussions
- Latest Newswire
- Archives
- Submit Story
- Story HOWTO
- Syndicate
- E-mail Alerts
Pluggable Schedulers vs. Pluggable Security
|
|
|
Comments (27)
7 earlier comments in the full thread.
The part about schedulers isn't the bit that has me confused, but security. So Linus has decreed "LSM is in the kernel whether anyone else wants it or not, and in spite of the security pros who say it's not a good idea" is how it sounds to me. He rants about the security people not understanding filenames and inodes, ok, that's something Linus knows inside out. But the rest of the security discussion- it's all a confusing mess.
Quoting:I can do a better job of deciding what you need on your computer even though I don't know the mix of work you run and don't know which things are most important to you.
If you understood the argument, you'd see that pluggable schedulers are not the answer to the problem. They are fundamentally flawed. By definition. He doesn't say what the right solution is (aside from the mythical "perfect" scheduler). Just that pluggable schedulers aren't it.
@tuxchick: My understanding of Linus' post is that schedulers are a technical problem. Security is a policy problem. You can solve technical problems but not policy problems. Therefor, pluggable security is a necessary evil.
>If you understood the argument, you'd see that pluggable schedulers are not the answer to the problem.
I understood the argument and I don't see that at all, although that depends on the definition of pluggable you use.
A true plug-in, something you swap in and out at will seems like a very bad idea. A scheduler is at the very heart of your system. You don't want instability. You don't want overhead. Both seem likely by-products of that kind of pluggability.
A compile time choice, however, can address the desire without adding needless overhead or instability.
From my standpoint, compile time choices would address most of the reasons behind the original selection.
Obviously, it wouldn't do much for a machine that keeps busy 24 hours a day, with very different workloads day and night.
I understood the argument and I don't see that at all, although that depends on the definition of pluggable you use.
A true plug-in, something you swap in and out at will seems like a very bad idea. A scheduler is at the very heart of your system. You don't want instability. You don't want overhead. Both seem likely by-products of that kind of pluggability.
A compile time choice, however, can address the desire without adding needless overhead or instability.
From my standpoint, compile time choices would address most of the reasons behind the original selection.
Obviously, it wouldn't do much for a machine that keeps busy 24 hours a day, with very different workloads day and night.
> You will get what I care about and like it or else.
And when has Linus' argument been anything else?
What I hope comes out of this in the long run is a truly _tunable_ scheduler. Then it wouldn't be a matter of changing the code, but altering the parameters. I forget who was it on LXer who suggested that first, they mentioned it came from the mainframe environment. With IBM involved, it should be possible.
Linus has said that the desktop is where interesting things are happening. That doesn't sound so much like "I know what's best for you" so much as "Get it right, then talk to me about it."
And when has Linus' argument been anything else?
What I hope comes out of this in the long run is a truly _tunable_ scheduler. Then it wouldn't be a matter of changing the code, but altering the parameters. I forget who was it on LXer who suggested that first, they mentioned it came from the mainframe environment. With IBM involved, it should be possible.
Linus has said that the desktop is where interesting things are happening. That doesn't sound so much like "I know what's best for you" so much as "Get it right, then talk to me about it."
> I forget who was it on LXer who suggested that first, they mentioned it came from the mainframe environment.
I know that I have mentioned my days as a performance tuner/capacity planner in big mainframe shops and how I longed wistfully for the control I had in those days.
I know that I have mentioned my days as a performance tuner/capacity planner in big mainframe shops and how I longed wistfully for the control I had in those days.
> ...and how I longed wistfully for the control I had in those days.
That would be it, then. I had not perceived from the discussions that such things could be tuned "on the fly", but had to be hard-coded.
Linus had said "schedulers are boring", but it sure has turned out to be an interesting series of discussions on the subject!
That would be it, then. I had not perceived from the discussions that such things could be tuned "on the fly", but had to be hard-coded.
Linus had said "schedulers are boring", but it sure has turned out to be an interesting series of discussions on the subject!
>That would be it, then. I had not perceived from the discussions that such things could be tuned "on the fly", but had to be hard-coded.
Oh my no. We could change things by editing configuration files and we could sit down at the system console and change them via real-time commands.
Oh my no. We could change things by editing configuration files and we could sit down at the system console and change them via real-time commands.
Hahaha! Those systems consoles.
My first day on the job as an operator on a multiple-ES9000 system(s), the cadre of suits came in, and one of the systems programmers leaned over the console and typed a command.
"Woops..." she said.
A high-pitched wail issued from back within the sea of blue-and-white cabinets on the raised floor. A lonely, steady pezo-beep without the beep part.
I sat there, dumb (a nod toward tuxchick), at the operator station, no clue how to react and dreading even the possibility that they might ask me to do something about it. Luckily, they all knew who I was even though I didn't know them, and a couple years later we all had a laugh about it when I mentioned how clearly I still remembered that empty feeling.
The system programmer mentioned that it really was important to not leave a dangling "-" in a "vary offline 220-" command. :^)
My first day on the job as an operator on a multiple-ES9000 system(s), the cadre of suits came in, and one of the systems programmers leaned over the console and typed a command.
"Woops..." she said.
A high-pitched wail issued from back within the sea of blue-and-white cabinets on the raised floor. A lonely, steady pezo-beep without the beep part.
I sat there, dumb (a nod toward tuxchick), at the operator station, no clue how to react and dreading even the possibility that they might ask me to do something about it. Luckily, they all knew who I was even though I didn't know them, and a couple years later we all had a laugh about it when I mentioned how clearly I still remembered that empty feeling.
The system programmer mentioned that it really was important to not leave a dangling "-" in a "vary offline 220-" command. :^)
>The system programmer mentioned that it really was important to not leave a dangling "-" in a "vary offline 220-" command. :^)
You might even say it's vary important.
You might even say it's vary important.
Quoting:I understood the argument ... A true plug-in, something you swap in and out at will seems like a very bad idea ... A compile time choice, however, can address the desire without adding needless overhead or instability.
I don't think you do. Linus' entire argument is against install/compile-time schedulers, because whatever scheduler you pick at compile time, it will always be the wrong one for some jobs. He then adds that, if you try to solve the install-time problem by making it dynamically plugable, that you end up with code that does the same as a single "perfect" scheduler, only vastly more complex.
He also states that pluggable schedulers will not satisfy the desire for control over job scheduling. You get control by changing the parameters of the scheduler, not the logic.
To make a nice analogy: pluggable schedulers are like having three car engines at home. One for slow-speed/high-pulling-power, one middle-of-the-road and one for high-speed/low-pulling-power. So everytime you change your driving behaviour (city trips, long distance highway, pulling a caravan) you need to take out the engine and put a different one in. What you want instead is a gear box - i.e. a tunable scheduler.
Quoting:What I hope comes out of this in the long run is a truly _tunable_ scheduler. Then it wouldn't be a matter of changing the code, but altering the parameters.
That is exactly what Linus is aiming for. Biggest problem today is that the tuning happens mostly automagically and that adjusting the scheduler manually is hackish.
Bill Davidsen:
Quoting:on desktops, response is (and should be king), while on a server, like nntp or mail, I will happily go from 1ms to 10sec for a message to pass through the system if only I can pass 30% more messages per hour
Linus:
Quoting:Bill, that's a *tuning* issue, not a scheduler logic issue.
...
most of the time we should aim for those things to auto-tune (we do have the things in /proc/sys/kernel/, but I really hope very few people use them other than for testing or for some extreme benchmarking - at least I don't personally consider them meant primarily for "production" use).
> it will always be the wrong one for some jobs.
Ah...control freaks of a feather hang together.
The problem isn't that a scheduler is wrong for some jobs. The problem is that Linus wants to control which jobs a scheduler is good for and dictate to the world that they will share his priorities.
PS: I spent ten years doing performance and capacity planning in some of the world's largest IBM mainframe centers. I also have my name on US and international patents for a work dispatcher/load balancer for a distributed call center system. Based on that background, I would, in all humility, like to think that I possess some modest comprehension of the tradeoffs involved in dispatching work.
Ah...control freaks of a feather hang together.
The problem isn't that a scheduler is wrong for some jobs. The problem is that Linus wants to control which jobs a scheduler is good for and dictate to the world that they will share his priorities.
PS: I spent ten years doing performance and capacity planning in some of the world's largest IBM mainframe centers. I also have my name on US and international patents for a work dispatcher/load balancer for a distributed call center system. Based on that background, I would, in all humility, like to think that I possess some modest comprehension of the tradeoffs involved in dispatching work.
I'm not saying you don't know your schedulers or job control. I'm saying you're not understanding the point Linus is trying to make.
And how did you have control? By tuning the scheduler or by installing a different kernel with different scheduler logic?
Quoting:I know that I have mentioned my days as a performance tuner/capacity planner in big mainframe shops and how I longed wistfully for the control I had in those days.
And how did you have control? By tuning the scheduler or by installing a different kernel with different scheduler logic?
>By tuning the scheduler
Definitely by tuning, but that's a matter of mechanism and implementation. I believe that I have often expressed a preference for more tunability, but I'm pretty easy as to means if the ends are accomplished.
After all, the Unix way of doing things and the mainframe way are very different, yet each manages to reach its destination.
Definitely by tuning, but that's a matter of mechanism and implementation. I believe that I have often expressed a preference for more tunability, but I'm pretty easy as to means if the ends are accomplished.
After all, the Unix way of doing things and the mainframe way are very different, yet each manages to reach its destination.
Quoting:I'm pretty easy as to means if the ends are accomplished.
And Linus' point is that pluggable schedulers are not the right means to this end. Tunability is. And so far the only people arguing in favour of pluggable schedulers are people who are interested in the same kind of control you are. So far, nobody has given any other reason why a pluggable scheduler would be a good idea.
>So far, nobody has given any other reason why a pluggable scheduler would be a good idea.
Actually, there has been a very good argument made, but it is derived rather than direct:
People have actually been willing to code up alternative schedulers and try them out, ie Con's and Hey!! now we've got CFS.
But I haven't heard of anybody doing squat WRT a tunable scheduler. I'm not in the loop for kernel-y things, so there might be an absolute firestorm of activity going on, but...
You know, I seem to recall that, when Linux came out, the conventional wisdom (cough, HURD) was that any new OS should be built on a micro-kernel, that monolithic OS's were a bad idea. They might have been right, but it doesn't matter: here we have the very-popular monolithic Linux.
Why do we have it? Because the people who were clucking weren't the people who were doing.
In my book, some sort of pluggable scheduler that gets coded up and refined beats the crap out of tunable scheduler that gets talked about and never implemented.
Actually, there has been a very good argument made, but it is derived rather than direct:
People have actually been willing to code up alternative schedulers and try them out, ie Con's and Hey!! now we've got CFS.
But I haven't heard of anybody doing squat WRT a tunable scheduler. I'm not in the loop for kernel-y things, so there might be an absolute firestorm of activity going on, but...
You know, I seem to recall that, when Linux came out, the conventional wisdom (cough, HURD) was that any new OS should be built on a micro-kernel, that monolithic OS's were a bad idea. They might have been right, but it doesn't matter: here we have the very-popular monolithic Linux.
Why do we have it? Because the people who were clucking weren't the people who were doing.
In my book, some sort of pluggable scheduler that gets coded up and refined beats the crap out of tunable scheduler that gets talked about and never implemented.
The existing scheduler already is tunable. It even autotunes itself. It's just not manually tunable because it lacks an interface. That's all. There is no need to "make it tunable". All someone needs to do is build a decent interface for it (something better than the existing /proc/sys/kernel/ interface).
> You know, I seem to recall that, when Linux came out, the conventional wisdom (cough, HURD) was that any new OS should be built on a micro-kernel, that monolithic OS's were a bad idea.
Having used OS-9, which was a sort of pre-micro-kernel micro-kernel, I'd say they were correct, and that a micro-kernel is better. However, that's not what Linus did or wanted to do. I believe Minix implements a micro-kernel, if anyone wants to compare them.
A very close analogy can be drawn to the RISC/CISC debates. Everyone agreed that RISC was better, but CISC seems to have won the war by adopting all of the best RISC ideas. It seems to me that the Linux kernel has largely adopted most of the concepts of a micro-kernel (loadable modules, most things being in user space, etc.) and implemented them in its monolithic kernel. They're even trying to implement user space device drivers from what I understand.
Having used OS-9, which was a sort of pre-micro-kernel micro-kernel, I'd say they were correct, and that a micro-kernel is better. However, that's not what Linus did or wanted to do. I believe Minix implements a micro-kernel, if anyone wants to compare them.
A very close analogy can be drawn to the RISC/CISC debates. Everyone agreed that RISC was better, but CISC seems to have won the war by adopting all of the best RISC ideas. It seems to me that the Linux kernel has largely adopted most of the concepts of a micro-kernel (loadable modules, most things being in user space, etc.) and implemented them in its monolithic kernel. They're even trying to implement user space device drivers from what I understand.
>The existing scheduler already is tunable.
To an extent, yes, and some of the remaining work may not be entirely scheduler-specific. Some of it may even be ignorance on my part -- which implies ignorance on the part of many people given that some folks have gone so far as to write alternate schedulers, an unlikely activity if the tuning info were easily available.
At any rate, some things I don't know:
How well does prioritization of CPU cycles map to prioritization of I/O? Protection of resident memory? How easy is it to assign certain tasks to the "I matter so don't mess with me" classification? How easy is it to assign tasks to the "He doesn't even want me to run, but hasn't managed to find the three-levels deep script that starts me up, so make me struggle for any cycle I can find and don't care if I get 99% degraded and 90% paged out" class?
Stuff like that. If it's all there but no mere mortals can find it, it might as well not be there.
To an extent, yes, and some of the remaining work may not be entirely scheduler-specific. Some of it may even be ignorance on my part -- which implies ignorance on the part of many people given that some folks have gone so far as to write alternate schedulers, an unlikely activity if the tuning info were easily available.
At any rate, some things I don't know:
How well does prioritization of CPU cycles map to prioritization of I/O? Protection of resident memory? How easy is it to assign certain tasks to the "I matter so don't mess with me" classification? How easy is it to assign tasks to the "He doesn't even want me to run, but hasn't managed to find the three-levels deep script that starts me up, so make me struggle for any cycle I can find and don't care if I get 99% degraded and 90% paged out" class?
Stuff like that. If it's all there but no mere mortals can find it, it might as well not be there.
One reason that even RMS uses to explain why HURD isn't ready so many years after it was begun, is because it turns out that message passing is a royal bitch to debug.
Minix works because it is an exceedingly simple implementation, a "student's breadboard" as it were.
Minix works because it is an exceedingly simple implementation, a "student's breadboard" as it were.
Quoting:which implies ignorance on the part of many people given that some folks have gone so far as to write alternate schedulers, an unlikely activity if the tuning info were easily available.
I don't think it's ignorance. They're probably just paining the bikeshed: http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.ht... (quote below).
Trying to create a new scheduler optimized for one single tasks isn't very difficult (compared to hacking other portions of the kernel) and technically a lot more interesting than doing the drudge work required to make the existing (and more complex) scheduler properly manually tunable.
Quoting:“What is it about this bike shed?” Some of you have asked me.
It is a long story, or rather it is an old story, but it is quite short actually. C. Northcote Parkinson wrote a book in the early 1960s, called “Parkinson's Law”, which contains a lot of insight into the dynamics of management.
[snip a bit of commentary on the book]
In the specific example involving the bike shed, the other vital component is an atomic power-plant, I guess that illustrates the age of the book.
Parkinson shows how you can go into the board of directors and get approval for building a multi-million or even billion dollar atomic power plant, but if you want to build a bike shed you will be tangled up in endless discussions.
Parkinson explains that this is because an atomic plant is so vast, so expensive and so complicated that people cannot grasp it, and rather than try, they fall back on the assumption that somebody else checked all the details before it got this far. Richard P. Feynmann gives a couple of interesting, and very much to the point, examples relating to Los Alamos in his books.
A bike shed on the other hand. Anyone can build one of those over a weekend, and still have time to watch the game on TV. So no matter how well prepared, no matter how reasonable you are with your proposal, somebody will seize the chance to show that he is doing his job, that he is paying attention, that he is here.
In Denmark we call it “setting your fingerprint”. It is about personal pride and prestige, it is about being able to point somewhere and say “There! I did that.” It is a strong trait in politicians, but present in most people given the chance. Just think about footsteps in wet cement.
--Poul-Henning Kamp on freebsd-hackers, October 2, 1999
Post a comment
A free account is needed to post. Write your comment here first: your text is kept in this browser while you sign up or log in, and is waiting for you when you come back.
Your message:Markdown: **bold**, *italic*, [text](https://link), - lists, > quotes, $x^2$ for math. Formatting help
Sign up and post (it is free) or Log in and post