Kernel Mode-setting
Kernel mode-setting (KMS) means that the kernel is responsible for setting up and changing the display mode: resolution, frequencies and color depth. Kernel modesetting and memory management describes some aspects of KMS along with Fedora specific information. KMS as a Linux feature was introduced in 2.6.29.
KMS in Nouveau is supported and recommended. It automatically offers nouveaufb, an integrated (into the DRM driver) framebuffer driver for the virtual console. Most notably this gives you a high resolution text console. More information about framebuffer console is in Documentation/fb/fbcon.txt. KMS and nouveaufb are inseparable.
Your boot time virtual console display driver affects the compatibility of nouveaufb:
VGA text mode: works
vesafb: works
efifb: should work
offb: works after fixes in offb (merge status unknown)
nvidiafb: does not work (does not make sense, either)
anything else: probably does not work
Activating KMS
First, make sure your kernel configuration has CONFIG_FRAMEBUFFER_CONSOLE enabled. Most sane configurations have it built into the kernel, but you can have it as a module (fbcon.ko), too. In that case make sure this module is loaded before or along with nouveau.ko. Otherwise your virtual console appears frozen.
There are two ways to activate KMS in Nouveau:
pass the argument modeset=1 to the Nouveau kernel module, e.g. $ modprobe nouveau modeset=1
set KMS to be activated by default in the kernel configuration (CONFIG_DRM_NOUVEAU_KMS)
When the Nouveau kernel module loads, it will change the display mode to the monitor's native mode and activate nouveaufb.
Am I running KMS?
If you cannot tell from the screen right away (you are not sure what a framebuffer mode looks like, or you had another fb driver active), look into /proc/fb. If it mentions nouveaufb, you are running KMS.
The Nouveau DDX also mentions in the X server log if KMS is in use. For instance, in /var/log/Xorg.0.log you may find a line:
(--) NOUVEAU(0): [drm] kernel modesetting in use
Deactivating KMS and unloading Nouveau
The framebuffer console reserves nouveaufb and therefore you cannot directly unload the Nouveau kernel module. You have to unbind nouveaufb first. When you unbind nouveaufb, the earlier fb driver or the VGA console driver will take over. Your kernel configuration needs to have CONFIG_VT_HW_CONSOLE_BINDING enabled.
NOTE: For NV50 class hardware and above, the VGA text mode cannot be restored, you will be without a working virtual console until you load a new framebuffer driver! For older hardware the VGA text mode console should be restored fine.
Here is an example script to unload Nouveau KMS:
#!/bin/bash echo 0 > /sys/class/vtconsole/vtcon1/bind rmmod nouveau /etc/init.d/consolefont restart rmmod ttm rmmod drm_kms_helper rmmod drm
The echo line unbinds nouveaufb from the framebuffer console driver (fbcon). Usually it is vtcon1 as in this example, but it may also be another vtcon*. See /sys/class/vtconsole/vtcon*/name which one of them is a "frame buffer device".
The console font trick may be needed with the VGA text mode to restore proper console state (to keep the active line visible), and depends on your distribution.