No, you'd just need to remove the HID class drivers, for example. So disallow HID_DEVICE, HID_DEVICE_SYSTEM_MOUSE and HID_DEVICE_SYSTEM_KEYBOARD. This would, if I'm following Coleman's link right, ...