The secrets contained in drivers and firmware aren't valuable, merely dangerous. Having the source to a graphics driver doesn't make it much easier to design graphics hardware, but it does make it a lot easier to identify and prove patent infringement, and it makes it easier to circumvent arbitrary restrictions that exist for purposes like product segmentation or DRM.