Puget Systems print logo
https://www.pugetsystems.com
Read this article at https://www.pugetsystems.com/guides/526
Dr Donald Kinghorn (Scientific Computing Advisor)

Using RHEL / CentOS 6.5 with the Intel Xeon Phi

Written on December 11, 2013 by Dr Donald Kinghorn
Share:

Can you use the new RHEL/CentOS 6.5 release with the Xeon Phi ... yes! But, there is a gotcha that we will need to work around. Read on.

 

I had been anxiously awaiting the CentOS 6.5 rebuild of the RHEL 6.5 release because I was hopeful that the new kernel patches would have fixes for some "Haswell" graphics trouble I've been having under the 6.4 release. Good news! Intel HD 4600 graphics on the "Haswell" processors is working fine now with CentOS 6.5. No more lockups after entering low power states. No more Intel nic spewing pause packets. No more shutdown/restart issues. Yea! That was a concern of mine because I wanted to use the CPU graphics for the Peak mini instead of an add-in graphics card as a work-around for the problems listed above. But, would there be any trouble doing a Xeon Phi MPSS setup and kernel module rebuild? ...

Unfortunately the answer to the last question in the paragraph above is, yes. I can't fault Intel for this since CentOS 6.5 is not officially supported for the Phi as of the date of this post (mid Dec. 2013). Note that I was working on a Xeon Phi setup for the Peak mini which is a unique platform for Phi, however, the problem I encountered with the mpss kernel module will apply in general to anyone trying a Phi setup with CentOS 6.5.

The problem? ... missing header files in the 2.6.32-431.el6.x86_64 kernel source

 

I did a fresh CentOS 6.5 install and then downloaded mpss-3.1.1-rhel-6.4.tar from the Intel MPSS page and then, with fingers crossed, proceeded with the kernel module rebuild so I could do the install. ... cd mpss-3.1.1/scr/ ... OK, lets rebuild the kernel module ...


[puget@mini src]$ rpmbuild --rebuild mpss-modules-3.1.1-1.el6.src.rpm
Installing mpss-modules-3.1.1-1.el6.src.rpm
... bunch of expected output ...

+ make KERNEL_VERSION=2.6.32-431.el6.x86_64 KERNEL_SRC=/lib/modules/2.6.32-431.el6.x86_64/build 'CC=gcc ' 'LD=ld ' MIC_CARD_ARCH=k1om
make -C /lib/modules/2.6.32-431.el6.x86_64/build M=/home/puget/rpmbuild/BUILD/mpss-modules-3.1.1 modules \
... so far so good ... then ...

In file included from /home/puget/rpmbuild/BUILD/mpss-modules-3.1.1/include/micint.h:62,
                 from /home/puget/rpmbuild/BUILD/mpss-modules-3.1.1/host/acptboot.c:44:
include/drm/drmP.h:76:21: error: drm/drm.h: No such file or directory
include/drm/drmP.h:77:27: error: drm/drm_sarea.h: No such file or directory
In file included from /home/puget/rpmbuild/BUILD/mpss-modules-3.1.1/include/micint.h:62,
                 from /home/puget/rpmbuild/BUILD/mpss-modules-3.1.1/host/acptboot.c:44:
include/drm/drmP.h:438: error: expected specifier-qualifier-list before 'drm_magic_t'

... bunch of errors and then more missing header files ...

include/drm/drm_crtc.h:35:26: error: drm/drm_mode.h: No such file or directory
include/drm/drm_crtc.h:37:28: error: drm/drm_fourcc.h: No such file or directory

... fail! ...

make[2]: *** [/home/puget/rpmbuild/BUILD/mpss-modules-3.1.1/host/acptboot.o] Error 1
make[1]: *** [_module_/home/puget/rpmbuild/BUILD/mpss-modules-3.1.1] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.32-431.el6.x86_64'
make: *** [modules] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.MJ6ayd (%build)

Crud! It seems drm.h, drm_sarea.h, drm drm_mode.h, and drm_forucc.h are not being included. Let's take a look at the kernel source directories;


[root@mini src]# cd /usr/src/kernels/2.6.32-431.el6.x86_64/include/drm/
[root@mini drm]# ls
drm_backport.h  drm_crtc.h         drm_encoder_slave.h  drm_hashtab.h   drm_os_linux.h  drm_usb.h    ttm
drm_buffer.h    drm_crtc_helper.h  drm_fb_helper.h      drm_memory.h    drm_pciids.h    i2c
drm_cache.h     drm_dp_helper.h    drm_fixed.h          drm_mem_util.h  drmP.h          i915_drm.h
drm_core.h      drm_edid.h         drm_global.h         drm_mm.h        drm_sysfs.h     intel-gtt.h

Nope, those headers are not in there! ... but they are in the system include directory;


[root@mini drm]# ls /usr/include/drm/
drm_fourcc.h   drm_mode.h     i810_drm.h     mga_drm.h      r128_drm.h     savage_drm.h   via_drm.h      
drm.h          drm_sarea.h    i915_drm.h     nouveau_drm.h  radeon_drm.h   sis_drm.h

Here's the workaround


[root@mini src]# cd /usr/src/kernels/2.6.32-431.el6.x86_64/include/drm/

[root@mini drm]# ln -s /usr/include/drm/drm.h drm.h
[root@mini drm]# ln -s /usr/include/drm/drm_sarea.h drm_sarea.h
[root@mini drm]# ln -s /usr/include/drm/drm_mode.h drm_mode.h
[root@mini drm]# ln -s /usr/include/drm/drm_fourcc.h drm_fourcc.h

I just put symbolic links to the system header files in the kernel source directory. Lets try the kernel module build again...


[puget@mini src]$ rpmbuild --rebuild mpss-modules-3.1.1-1.el6.src.rpm
Installing mpss-modules-3.1.1-1.el6.src.rpm
... bunch of expected output ...
... and then the glorious last line, ...

+ exit 0

Now we just move our newly built modules into the mpss install directory (replacing the old ones) (... Yes, I use rsync instead of cp, just a habit ...)


[puget@mini] rsync -av rpmbuild/RPMS/x86_64/mpss-modules-*  mpss-3.1.1/

Then do the install;


[root@mini mpss-3.1.1]# yum install --nogpgcheck --noplugins --disablerepo=* *.rpm

... and then finish up the install and configuration. After several hours of testing everything seems to be working great.

Happy computing! --dbk

Tags: Intel, Xeon Phi, HPC
Comments