Puget Systems print logo


Read this article at https://www.pugetsystems.com/guides/508
Dr Donald Kinghorn (Scientific Computing Advisor )

Top 5 Xeon Phi Misconceptions

Written on October 3, 2013 by Dr Donald Kinghorn

The Xeon Phi has been generally available for over 6 months now and most people that keep up with developments in high performance computing have heard about it. However, I've talked with many people that have heard about the Xeon Phi but have little understanding of its characteristics other than it's a many core co-processor that is being used in some of the worlds fastest computer systems. What follows are the top 5 misconceptions I've been hearing from people about the Phi.

1)"If I add a Phi card to my system I will have an extra 60-cores for my programs to run on."

Well, yes and no. The Xeon Phi does have 57-61 cores and they are 64-bit x86 compatible. However, those Xeon Phi cores look like they belong to a separate computer connected over an internal network to your host OS. I've talked with several people who thought those extra cores were just automatically available to your host system. Those extra cores do not just show up along side your system processor cores and are not directly visible to the host OS!

2)"The Xeon Phi is Linux only."

Again, this is yes and no. The Phi itself runs an embedded Linux OS in memory on the card. The card has a boot-loader flashed onto non-volatile memory and when the card starts up it loads a file system and Linux kernel that are stored on the host system. It's called uOS. However, the card can be booted into uOS from a Windows host! There is a Windows driver for the card and it works fine. I've tried it. (I'll talk about that in another post.) Just keep in mind that even on a Windows host the card is still running Linux. You can't run Windows programs on it since the executables would be binary incompatible.

3)"You can run any Linux programs on the Phi."

NO! As I said earlier the Phi is running an embedded Linux OS called uOS. The "u" is really supposed to be a Greek letter mu and stands for "micro", as in micro-OS. That's a hint! Most of the OS functionality on the card is provided by a cool application called "busybox" which provides some of the functionality of many common core Linux command line system programs in a very compact way. It's a pretty lean Linux system! Core system level commands and network capability are provided on the card but not much else. You can directly login to the card and run programs in "native mode" but, any programs you run are going to need to to bring any needed run time libraries over to the card or make them available over the network interface to the card. (You can do NFS mounts to the card.) It's really not a general purpose Linux OS. It's an embedded system OS running in, and using up, the limited memory available on the card. ... I've had more than one person ask me if they could run Windows programs on the Phi using "Wine", sorry, that's not going to work!

4)"It's easy to write programs for the Xeon Phi"

No way! Programming is not easy period. It is easier to get something to run on the Phi than it is on a GPU with CUDA for example. However, getting a code to run with good performance on the Phi requires a considerable effort. You have to be able to utilize all those cores and hardware threads along with the core vector units in order to get good performance out of the Phi. Also, you have the same problem of limited memory and limited bandwidth over the PCI bus that you have when working with GPU accelerators. It's a non-trivial programming effort. Is it worth the effort? Yes. The effort you put into optimization for the Phi will also benefit your codes performance on the CPU and the Intel compiler tools help tremendously with this.

5)"The Phi is just a fad, it wont last"

I'm calling that a misconception too. Those who have been in the HPC business for a while may have "déjà vu" moments when they look at Phi. Remember the "Cell BE"? (the "Cell" processor was used in the PlayStation3) It had interesting potential, it was added to some super-computers of the time, and effort was made to take advantage of it. It didn't go well! The architecture was too limiting. The Phi does not have this problem! How about FPGA's (field programmable gate arrays)? Tremendous potential, and they are still actively developed and advances in usability have been made. However, they are by design special purpose and can't really be used as a general platform for running varied codes. Phi has a big advantage in this regard. Then there's Tesla + CUDA. This is the strongest competition for the Xeon Phi and has a solid 4 year head start. Nvidia has done an outstanding job at nurturing a developer "ecosystem" and there are many successful projects making good use of the strong compute capabilities of their GPUs. However, writing compute intensive code to run well on GPUs is challenging and requires a solid understanding of the architecture. Tesla isn't going away any time soon and the Phi is not going to replace it. The Phi is different from any compute accelerators that have come before and I believe it is going to evolve into a stand alone platform. It's nearly a general purpose many core "CPU" already and Intel has the means to move it forward to become a complete high performance computing system. The next couple of years should be interesting indeed!

Tags: Intel, Xeon Phi, HPC

I found this column informative, but confusing. How can one be sure that it isn't another chip that will not succeed on the market if it can't be easily used in the most obvious ways? Actually, that question is easy to answer, but no hint of how you could actually put the Xeon Phi to work for you is given here.

Posted on 2014-08-24 17:16:47
Donald Kinghorn

... you are wise to keep history in mind when you look at things like Phi, I've been around long enough to have seen many "co-processors" fail and there is still really good stuff like FPGA's that haven't caught on well because of usability issues. However, this is always going to be the case when people are seeking the highest performance, you have to make trade offs. Right now the Phi is for developers, not so much for end users! Parallel programming is hard no matter what the underlying hardware is. The nice thing about the Phi is that techniques you use for optimal performance on a modern multi-core CPU is the same thing you need for the Phi, (thread management and vectorization) Also, the Phi is not going away! Mid year 2015 will be a game changer with Knights Landing and eventually it will support full "standard" OS installs and behave like a "normal" system ... In the mean time, new conventional CPU stuff coming out of Intel is fantastic! Best wishes -Don

Posted on 2014-08-25 16:22:10
Cheng Tan

hi dear donald,
i want to ask u does the phi support message passing in its architecture? it has coherent l2, so i believe it is based on cache coherent, then the pthread programming would do well in this shared memory environment. but does it support message passing with the support of special message passing buffers like the intel scc?

Posted on 2014-10-15 17:03:30

Is there anyway this could be used in a gaming rig?

Posted on 2014-10-27 12:59:28

Hello Dr. Kinghorn
I've seen a Xeon Phi priced at an insane $200. First, however, I need your help.
I am not a programmer. What applications use Xeon Phi? I am interested in Stata and Autodesk software. Supposing that Stata uses Xeon Phi, how would I run it from windows? Or would I need to dual-boot linux? If so, how much RAM should I give it, out of 32 GB?

Posted on 2015-01-17 14:06:34

You've made several erroneous statements:

"Most of the OS functionality on the card is provided by a cool application called "busybox" which provides some of the functionality of many common core Linux command line system programs in a very compact way. It's a pretty lean Linux system!"

First off, BusyBox isn't an application, it's a userland and micro-kernel.

BusyBox / Linux, is a stripped down Linux kernel with the Busybox command set, support libraries, support scripts and binaries necessary to build a distribution around said userland and kernel. The most common userland associated with the Linux kernel is GNU / Linux for which the vast majority of the distributions are based off of. I can see why people call it an application but it's not really.

Technically speaking; an application requires an operating system and support libraries in order to work (kernel level wise). That's where you're error is. BusyBox is the kernel and utilities in one file but in and of itself isn't an application. The micro operating system or uOS for short is in fact a distribution built around the BusyBox binary and userland. Binaries that are loaded in from the PCI-E bus and executed in the userspace of the uOS are the "applications".

Posted on 2016-03-24 07:20:36

Small question: you wrtoe, that u cannot use a Phi as a/the mainprocessor in the System, but how do you explain this one???? :


as there is clearly shown, there is no extra CPU or even a Grid Array for on to be placed. in this System the Phi is THE main processor!

P.S.: If you mean "micro" to be a greek letter then use that damn letter. on my Keyboard it's "ALT GR + M" et viola: Âu. But never ever use that smal u instead. if you cant write a "my / Âu" (even your 'mu' is a giant mistake, have a look at a Dictionary or at least Wikipedia) then use the word micro or the letter "m" as an abrivation of Micro.

Posted on 2016-11-24 13:19:52
John Doe

Interesting. Interesting indeed. I must go do some googling on this. I have never seen this before. It does make sense though that it would be socketed to a board anyhow. It is just that the Phi I knew of was a PCI express card. I am sure that it will be expensive at least for a while until something else comes along. I am looking for hardware for photogrammetry and that is very CPU intensive. It runs a CPU at 100% for anywhere from 20 minutes to an hour depending on the object you are making a 3D scan of. At least until they make it utilize the GPU more, it will be painfully slow on some systems.

Posted on 2017-03-07 06:35:38
John Doe

I am glad that I have read this. If I am not mistaken, these don't work on all motherboards too so keep that in mind when selecting the all the other stuff. I am glad that I have read all of these limitations before buying one of these expensive things. Even on eBay they are expensive used. If they ever design a plug and play version of this or anything similar to that I can imagine it becoming popular. There are other ways to get many processor cores. Try a motherboard that supports CPUs with high core counts. I have seen Opterons with as many as 16 cores. They will run your software directly and without having to use Linux and Busybox. Some Intel CPUs have a lot of cores also. Neither the AMD or Intel plug and play processors will be 60 cores but you could go with your favorite brand, AMD or Intel and choose a motherboard with 4 sockets that just happens to allow CPUs with many cores. I don't know how many cores the latest Xeons have but you could get 64 cores with 4 16 core Opterons and a motherboard that can handle them and all the RAM you will ever need for the rest of your life. Intel is pretty good too but I like a less expensive CPU.

Posted on 2017-03-07 06:24:35