Puget Systems print logo

https://www.pugetsystems.com

Read this article at https://www.pugetsystems.com/guides/490
Dr Donald Kinghorn (Scientific Computing Advisor )

Will your motherboard work with Intel Xeon Phi?

Written on August 6, 2013 by Dr Donald Kinghorn
Share:

             

24655 + 24656 = ???

Thinking about picking up a Xeon Phi coprocessor and taking a shot at many-core + SIMD parallel programming? Sounds like fun! Well, now what? There's a long list of things to consider but if you are thinking of just getting the card and trying to use it in an existing system you may be in for some disappointment if you expect it to work on any old motherboard.

First, Intel assumes you are plugging your new Xeon Phi into a dual socket Xeon E5 motherboard and they have a sensible reason for assuming that. Here's why. If you can't get your code to scale in parallel across the 8 to 16 cores of a dual E5 setup AND exploit the AVX vector instructions on the E5 then you aren't going to have much luck with the Phi. I can see the point but, if you are a developer and just want a more modest single CPU system, what then? Well, there are a couple of Sandy Bridge-E core i7, socket LGA 2011, X79 chipset boards that will work with Phi, (ASUS X79 boards note: "WS" version has support, others may not!). I'm looking at some newer "Haswell" boards now but I don't have any validated yet.

The biggest problem seems to be the requirement,

"BIOS support for memory mapped I/O address ranges above 4GB"

the Xeon Phi requires a motherboard and BIOS with large "Base Address Register" support!

What's that? When a system starts up one of the things the BIOS does is look for devices on the PCI bus. When it finds a device it queries for the "resources" being requested by the device. These resources can be a request for a memory mapped I/O (MMIO) address range. It then reserves the requested block of address and assigns a start address for the block to one of the devices base address registers. Those address are what the system uses to communicate with the device. The Xeon Phi requires that the BIOS provides a "large" i.e. 64-bit address for that.

Where to look for this in your BIOS

Start your system and hit the magic key that will drop you into the BIOS settings screens. (it may be the [Del] key or one of the F keys like F2 maybe ...) Look for *Advanced settings* and then look for  *PCI settings*. on an Intel board you may find something like; "Memory Mapped I/O above 4GB    [Enable]" on an ASUS board it will probably look like; "PCI 64bit Resource Handling Above 4G Decoding    [Enabled]" You might also see something referring to "MMIO above 4G" or maybe even "large BAR support". What you hope to find is *something* and an option to [Enable] it. It may or may not be enabled by default. If you don't find anything like this in your BIOS then all bets are off as to whether your Xeon Phi is going to work or not. Actually, my bet is that it wont work!

If you do have the right PCI 64bit resource support does that mean your board will work with Phi? NO! It's necessary but not sufficient.

Unfortunately I have tried boards that looked promising and had the right PCI settings but still did not fully assign resources correctly for Phi.

Here's a hint at why the Phi needs that large address space decoding.

Look at a few lines of output from  lspci -vv below for a Phi card,  

 

 

[kinghorn@phi1 ~]$ lspci -s 84:00 -vv 84:00.0 Co-processor: Intel Corporation Device 2250 (rev 11) 
         Subsystem: Intel Corporation Device 2500 
... 
         Region 0: Memory at 3c1c00000000 (64-bit, prefetchable) [size=8G] 
         Region 4: Memory at fbd00000 (64-bit, non-prefetchable) [size=128K] 
...

Take note of that [size=8G]!

Contrast that with the lspci output for an NVIDIA Titan card with 6GB
of memory,


[kinghorn@phi1 ~]$ lspci -s 03:00.0  -vv
03:00.0 VGA compatible controller: NVIDIA Corporation Device 1005 (rev a1) (prog-if 00 [VGA controller])
        Subsystem: NVIDIA Corporation Device 1035
...
        Region 0: Memory at 5a000000 (32-bit, non-prefetchable) [size=16M]
        Region 1: Memory at 50000000 (64-bit, prefetchable) [size=128M]
        Region 3: Memory at 58000000 (64-bit, prefetchable) [size=32M]
        Region 5: I/O ports at a000 [size=128]
...

I think that [size=8G] in there from the Phi lspci output is a big hint about why Phi has the larger BAR requirement! What you see in the Titan lspci output is more what I'm used to seeing, a modest " aperture"="" to="" the="" device="" memory.

The bottom line is, unless your vendor has validated a specific board for use with the Xeon Phi then don't expect it to work ... I wish it wasn't so.

 

Tags: Intel, Xeon Phi, HPC
Rakib Hasan

Hi Dr. Kinghorn,
Thank you very much for the article.
I saw that you said ASUS X79 boards will work.
I am thinking about building a machine with ASUS P9X79LE board with an
Intel Core i7-3820 Sandy Bridge-E 3.6GHz 4-core processor. Do you think
it will work?
Also, before buying, I was checking the manual for the motherboard but couldn't find that option. If you have that board, could you point out the path for that option in BIOS?

Posted on 2013-10-05 03:28:53
dkinghorn

Hi Rakib,

I looked at the manual for the P9X79LE and, as you
observed, there is no mention of "Above 4G decoding" which is what you
would usually look for in an ASUS BIOS.

THAT BOARD IS NOT GOING TO WORK!

However,
the P9X79 WS will work. In fact they mention Xeon Phi support in the
main feature list for the board. (unfortunately that is a much more
expensive board)

http://www.asus.com/Motherb...

I
was a little surprised that the LE version did not have the large BAR
support since it's basically the same chip set. ... but even if a
motherboard "could" work the guys writing the BIOS have to code it in.
I'll put a note in the post about this...

Best wishes-Don

Posted on 2013-10-06 03:50:10
Rakib Hasan

Hi Dr. Kinghorn,
Thank you for your reply.
I saw that this motherboard supports Xeon Phi 3100 series. But I still couldn't find the option in the BIOS for this motherboard. Could you tell me the path?

Also, the page says that "active fan SKU only" ( for Xeon Phi 3100 series).

Could you please explain what that means?

Posted on 2013-10-06 04:14:07
dkinghorn

... I think the "active fan SKU only" line is for liability reasons. That board is generally used in "workstations" rather than special rack chassis like our "Peak 2U". They don't want to be responsible for someone putting a passively cooled card in a "tower" case without special cooling for the card. I think a passive cooled Phi would go into thermal shutdown in a couple of seconds without adequate air moving through the card.

The motherboard manual for the P9X79 WS that is on the ASUS web site does not show the setting for Phi. However, we routinely grab BIOS screen-shots for support reference. The image below shows the screen with the line, "Intel Xeon Phi support". It is set to "Disabled" by default.

Asus P9X79-E WS: screenshots taken on 2013-09-13 16:50:19

Posted on 2013-10-07 23:34:57
dkinghorn

I'll add a little more... The board I tried, and am referring to above, is the P9X79-E WS board. The "-E" version has an extra PCIe slot and the BIOS manual lists in the specifications

"Up to 3 NVIDIA®Tesla cards and Intel®Xeon®phi cards"

However, it doesn't mention anything about "Above 4G decoding" or "Xeon Phi support" in the pages describing the BIOS settings. I just happened to have a screen-shot of the BIOS that shows that it does indeed have support. I don't have any screen-shots for the P9X79 WS, no "-E", but the ASUS web site has

"Support Intel® Xeon Phi™ 3100 series (active fan SKU only) and NVIDIA Tesla K20C cards."

on the main page for the motherboard (http://www.asus.com/Motherb... But again the manual on their site does not show any Phi support in the BIOS settings.

Is this confusing? YES! When I was doing research for this blog post I thought I would be able to find boards with "large BAR" support by looking in the BIOS manuals. I soon discovered that what vendors list in their paper or on-line pdf BIOS manuals is often NOT what is actually in the BIOS! I can partly understand this since BIOS software goes through lots of revisions during the life of a board but nobody ever really "wants" to update documentation. Especially documentation that goes into print (paper).

Posted on 2013-10-08 00:17:50
Rakib Hasan

Hi Dr. Kinghorn,
It took a while, but I finally managed to build a machine with the P9X79-E WS board and it works.
Thank you very much.

Posted on 2013-11-05 18:01:34
Benjamin Lowry

Just updated my bios for my asus Z9PE-16 2/L and enabled the 4g decoding and it appears to have worked. Thanks for this guide, message me if anyone needs help with this!

Posted on 2014-01-24 00:00:52
Mickhael Young

hello there mate i was wondering i've just bought the asus z9pe d8 ws mother board, and had no idea how the xeon phi works on it, and plus i heard it needs to be optimize to get the best out of it, and so how is it to optimize the xeon phi for autodesk 3ds max?

Posted on 2014-04-14 16:16:47
Donald Kinghorn

Hi Mickhael, you might want to read this
http://www.pugetsystems.com...
Programs have to be specifically written to work with Phi ... and yes, it's a good bit of work to get good performance. You could always send a note of encouragement to Autodesk about adding support for Phi
best wishes --dbk

Posted on 2014-04-14 19:08:18
Malcolm X

I have the same motherboard with e5-2620 and 128gb ram could you please send me steps you took to get phi to work thanks

Posted on 2015-02-18 06:38:00
Christo Cordier

Good day , i am looking to upgrade but ,before i fork out alot of money plz explain if the xeon phi is supported for by adobe premiere or sony vegas i cant find anything online specific to that
.

Posted on 2016-03-31 22:00:53

No, it won't work for Premiere or Vegas. Phi is basically a whole other computer running Linux within your system, so unless the program supports offloading to another computer (basically a cluster) it will not work with Phi. I would read over our Top 5 Xeon Phi Misconceptions post for a bit more info https://www.pugetsystems.co...

Posted on 2016-04-01 02:48:54
Christo Cordier

Thank you very much... I wish it was possible though...

Posted on 2016-04-01 02:51:54
Jeremy

Hi Dr. Kinghorn:
Thank you for a very insightful article. Nowadays, x79 mainboards are a rare find in my locale. The most common models are based on the z87 chipsets (socket 1150). Will motherboards built on this chipset work with Xeon Phi?

Posted on 2014-02-01 14:01:32
Donald Kinghorn

Hi Jeremy, I didn't get a notice of your post, sorry for the late reply ... The only z87 board I know of that has worked is the ASUS Gryphon that we used but it has a modified BIOS to get the large BAR support (above 4G decoding) ... and it's being replaced ... There is a Supermicro Q87 board that is supposed to have support for Phi but I haven't tried it. The best way to know about Phi support is to contact the board manufacturer directly.
best wishes --dbk

Posted on 2014-04-14 19:17:32
Gabriel Araujo

My Asus Maximus VI Extreme has (memory remap to enable or disable memory remap above 4G) Does it support Xeon Phi?

Posted on 2014-05-10 15:30:45
Donald Kinghorn

Hi Gabriel, I believe that is referring to being able to "use" system memory above 4GB. The support you need for Phi is on the PCI bus. ASUS uses the phrase "above 4G decoding" to describe this under PCI settings. It is actually referrign to large "Base Adress Register" support for on the PCI . It's worded badly in the BIOS manuals. ASUS boards that support Xeon Phi directly will say so explicitly in the advertising copy or in the manual explicitly.
Best regards --dbk

Posted on 2014-05-11 19:33:51
Gabriel Araujo

Thank you for your reply.

I researched more and found out that ASUS motherboard supports intel xeon phi. Is the board Z9PE-D8 WS, P8Z77 WS and P9X79 WS, support perfectly with Phi 3100 series and 5110P

Posted on 2014-05-12 12:32:09
andalib

hello i have macbook pro with thunderbolt port.is it possible to use intel xeon phi co-processor using external pcie chassis.please let me know

Posted on 2014-06-08 05:33:25
Donald Kinghorn

It could conceivably work but I think it's pretty unlikely. There are way too many things that could break/not work.
1) I don't know if thunderbolt supports 64-bit base address registers ( it might/should )

2) If 1 is OK then you still need a 3.x linux kernel for thunderbolt support
3) there is no Phi MPSS driver for 3.x kernels
4) you would have to run linux on your macbook (yuk!)

Posted on 2014-06-08 20:13:35
andalib

thanks Mr. Donald

thunderbolt supports 64-bit base address registers.but the thing is my macbook processor is core i7.i have seen in youtube intel guyz using xeon phi with intel xeon e5/e7 processor.i am just curious about xeon phi. if it works in my macbook then i can virtualize so many OS simultaneously

Posted on 2014-06-09 06:00:59
Donald Kinghorn

Two things:
It's not going to work with mac OS X OR thunderbolt at this point in time (I talk with Intel about thinderbolt latter this week ..., right now this kind of usage does not work with thunderbolt)
Second there is no way to use the card for virtualization the way your are thinking (i.e. you can't run virtual machines ON the Phi. ...)

Wait until the end of the year and look at it again, things may be different.
Best wishes --dbk

Posted on 2014-06-09 18:23:49
aung

Hi Dr. Kinghorn,

Thanks for the article.

Would like to know if ASRock X99M Extreme4 board ( with stock bios, not customised one ) work with Xeon Phi ?

Thanks.
-Aung-

Posted on 2014-11-21 15:18:48

No, a special BIOS from ASRock is needed for proper support of cards like the Phi. At some point that functionality may be rolled into a future BIOS update, but so far as I am aware that has not happened yet.

Posted on 2014-11-21 16:31:06
aung

Thanks for the reply.
May I know how can I get this special BIOS update ?

-Aung-

Posted on 2014-11-21 16:54:06

If you purchase a system from us with the X99M Extreme4 motherboard we can load the BIOS for you here. Aside from that, I would recommend contacting ASRock tech support.

Posted on 2014-11-21 19:16:29
aung

I see. Thanks for the info.

-Aung-

Posted on 2014-11-22 02:47:26
Grant Roy

Thanks very much for posting this article.

I don't suppose this would work? http://www.supermicro.com/m...

Anyone have any idea what the cheapest 1U or 2U server is that has a board for the xeon phi? Any help greatly appreciated.

Posted on 2014-12-31 05:02:35
Donald Kinghorn

Hi Grant, That board is a bit dated and I don't see anything in the BIOS manual ... however, it could be possible that Supermicro has added the large BAR support in a latter BIOS. You should probably ask them difectly. Puget has a nice 2U chassis that will take up to 4 Phi's but I'm not sure if that would be the cheapest. We are working up a small form factor system that will take 1 Phi. We have a 3D printed cooling duct for it :-) ...The new Peak Mini.
Best wishes
-Don

Posted on 2015-01-02 14:48:58
Grant Roy

Thanks Donald...the 1U with a 3d printed duct sounds pretty cool:) I'll look at the 2U chassis.

Posted on 2015-01-02 17:03:05
Guest

Any chance the Phi will work with a new AMD board?

Posted on 2015-01-22 11:58:28
Donald Kinghorn

Hi Koguma, very unlikely I'm afraid. I did put a Phi in a dual Opteron G34 based system once to see if it would be recognized ... it wasn't. However, the first issue you have to check is support for large BAR's. That's part of the PCI spec and there is no reason that I can think of that AMD chipsets would not be able to support that ... but they also, probably don't have much reason to support them either! Best wishes -Don

Posted on 2015-01-22 18:04:00
Guest

Well, I did my test on a Gigabyte GA-990FX-UD3(rev.4) board. It wouldn't even boot up. Played a nice bios tune on the internal speaker and refused to start up until I removed the card.

I'm now trying to find the cheapest board I can that can run the Phi. Do you by any chance sell just boards that can work with the card?

Posted on 2015-02-28 11:00:08
Fred

Any X99 motherboards with stock BIOS work with Phi's?

Posted on 2015-01-27 21:34:50
Eltharion

I have tested the X99-M WS (micro ATX) with success, stock bios updated to the latest version.

Posted on 2016-11-28 14:32:47
GasGeverij

Will 2 phi's be able to fit into this board ? Or will it simply not fit ?

Posted on 2018-01-03 03:25:32
Donald Kinghorn

That is the board we used to use in the Peak Mini. It will work fine with 2 active cooled Phi's
It is actually a really nice board. The full ATX version with the PLX PCIe switch has given us some occasional trouble but the '-M' is great.

Posted on 2018-01-03 16:54:11
Zibi Holka

Hi,

After 2 months of hard work finally I have managed to make my Phi working!

Works:
Hardware:
- ASUS GRYPHON (TUF) Z97 mATX motherboard! BIOS:2205 -standard BIOS downloaded form asus.com
- Intel Core i7 i7-4790K CPU (Quad Core 4GHz, Socket H3 LGA-1150)
- Corsair Platinum 16GB 2x8GB Memory Kit (CMD32GX3M4A2133C9)
- Xeon PHI 31s1p
- [I am sure you can change CPU+Mem. I just put my current config]

Software:
- Fedora 21
- Kernel (std. fedora): 3.18.7,
- MPSS: 3.4.2
- MPSS-Modules recompiled from https://github.com/xdsopl/m...
[Be carefull use Branch 3.4 not default one as they 3.2.3 and crashes when mpssd start]

dmesg message [Good]:
[ 0.213052] pci 0000:02:00.0: BAR 0: assigned [mem 0x800000000-0x9ffffffff 64bit pref]

Don't work:
RHEL6.5 with standard mpss modules
RHEL7.0 with standard mpss modules
as they can't handle PCI<->memory allocation

dmesg message [Bad]:
[ 0.214608] pci 0000:02:00.0: disabling BAR 0: [mem 0x00000000-0x1ffffffff 64bit pref] (bad alignment 0x200000000)

Notes:
1. ASUS BIOS below& 2101 doesn't keep setting PCI IOMMU>4G, after save, back to Disabled.
2. Module compilation will fail so to fix error:

vi host/linux.c
replace line:
rc = __f_setown(filp, task_pid(current), PIDTYPE_PID, 0);

with:
__f_setown(filp, task_pid(current), PIDTYPE_PID, 0);

[This is my change to make/compile work and seems to be working, I think dev drivers are for kernel 3.17]

3. no pci=noapic, pci=realloc needed do far.

4. lspci:

02:00.0 Co-processor: Intel Corporation Xeon Phi coprocessor 31S1 (rev 11)
Subsystem: Intel Corporation Device 2500
Flags: bus master, fast devsel, latency 0, IRQ 17
Memory at 800000000 (64-bit, prefetchable) [size=8G]
Memory at bf200000 (64-bit, non-prefetchable) [size=128K]
Capabilities: [44] Power Management version 3
Capabilities: [4c] Express Endpoint, MSI 00
Capabilities: [88] MSI: Enable- Count=1/16 Maskable- 64bit+
Capabilities: [98] MSI-X: Enable+ Count=16 Masked-
Capabilities: [100] Advanced Error Reporting
Kernel driver in use: mic
Kernel modules: mic_host

Hope it helps!

Zibi

Posted on 2015-02-22 20:07:16
Donald Kinghorn

Hi Zibi,
Nice job! You solved a lot a problems for that setup ... the more trouble you have the more you learn ... that's a nice "bleeding edge" configuration using Fedora 21, thanks for sharing. Also, I wasn't aware that ASUS had added the large BAR support to BIOS 2205, good!
Best wishes
-Don

Posted on 2015-02-24 17:29:28
lmanrique

Hi Zibi, thanks for share this tutorial, I'll follow your steps soon, can you please check if this motherboard is exactly the same of your?

http://www.asus.com/us/Moth...

Posted on 2015-03-06 07:46:48
Zibi

Yes that's motherboard which I use. PHI in PCI-GPU Slot as is directly wired to CPU.

Posted on 2015-03-10 09:57:50
Henk-Jan Lebbink

Hi Zibi,

Thanx for sharing this information. I also own a ASUS GRYPHON Z97, but I can't get the Xeon Phi to work on Windows 7. The Xeon Phi driver complains with "This device cannot find enough free resources that it can use. (Code 12)". I suspect the PCI busses are not using 64-bits. Also, I could find the reference to in the BIOS for "Above 4G decoding" but that did not do the trick, are there any other options that need to be set?

Questions:
1] Did you use the default settings in BIOS 2205? If not, what did you change to allow 64-bits MMIO.
2] What do you mean (in your previous remark) when you said "PHI in PCI-GPU Slot"?

Regards HJ!

Posted on 2015-06-24 08:59:41
Dirk

Sorry for bringing up this old thread again... Did you find a Solution for "This device cannot find enough free resources that it can use. (Code 12)" I have the same issue with the same config in Windows 7.

Regards
Dirk

Posted on 2015-10-05 22:06:30
Zibi Holka

Hi Sorry for laaaate reply as I didn't received any notification about your question. Hm I think it will be difficult to run on windows 7. Fingers crossed for you.

During my experiments I found:
1) BIOSES pre 2205 didn't hold "Above 4G decoding" at all. After Save & back to BIOS this feature was always disabled regardless of other settings. After flash 2205 finally 4G was working (Save&back to bios hold this setting). Just enable 4G at default bios settings, and as I remember no specific other bios settings were needed.
2) PCI-GPU slot -means for me no.1 (closest to CPU) In documentation called PCI3 3.0/2.0 x16_1 - at least this one is directly connected to the CPU

Doing my research I found that all conf is pushed by BIOS to CPU (chip) during init as all BARs registers are there. On earlier kernel (linux) versions didn't work as they couldn't allocate PCI memory. (started working for me starting from kernel 3.18) - This is pointing to Software-(,Driver,Kernel ) was not able properly handle PCI memory mapping, even if hardware is capable of doing it.

Not sure if Intel will provide proper drivers for Win 7 & (Enterprise) Linux'es as mine MPSS were modified and came from https://github.com/xdsopl/ .... Obviously oryginal Intel driver works perfectly on Enterprise linux (RHEL) and 'Supported' Hardware which is bit expensive.

Posted on 2016-04-10 16:37:51
Dirk

I switched to SUSE Linux and everything is running fine with two cards and two temp regulated but still noisy fans

Posted on 2016-05-05 19:57:59
William E Barnes Jr

Can a xeon phi be used WITH a tesla ?

Posted on 2015-05-13 05:17:03
Donald Kinghorn

Yes, I have done systems with both Phi and Tesla in the same box. To make your life easier in this situation I would recommend that you have a NVIDIA card for display too. AND, ALL of the normal headaches from doing a Phi or CUDA setup will apply. Also, you want to be sure that your system will support both i.e. you will need large BAR support in the BIOS for Phi (that usually doesn't cause any trouble for NVIDA cards but I can't guarantee that in any specific setup there wont be some annoying problems)

So, yes, but just realize that you could need to exercise your sys-admin chops at some point to have everything working well.
Best wishes -Don

Posted on 2015-05-14 16:10:44
sl1pkn07

Hi. any know if Xeion Phi 5110P can work with EVGA SR-2 motherboard? (chipset intel 5520)

greetings

Posted on 2015-08-18 19:21:47
Donald Kinghorn

... that looks like an older board ... my guess is, no. If this is a board in a system you are using then update the BIOS to the latest and then check to see if there is a setting for large BAR a.k.a "Above 4G decoding" That's the main show stopper for Phi. If you don't have that it wont work ... Best of luck --Don

Posted on 2015-08-19 18:17:03
StreamHPC

We created a list of motherboards here: http://streamcomputing.eu/b...

Posted on 2015-10-06 06:44:53
Wacky Tobaccy

Resurrecting this old thread, because. I finally got a motherboard, cpu, etc. To try out my old 31s1p that's been hanging around my desk for the past 2 years. I decided to try the Asus H97-Plus MB as it's the cheapest Asus MB that has 'Above 4GB encoding' in the bios. Lo and behold, it doesn't seem to be working. I've contacted Asus support to see if they can (I'm hoping) send me some kind of custom bios. Has anyone had any luck with this board? It seems like it should work...

Posted on 2016-04-18 06:24:02
Wacky Tobaccy

Just wanted to update people who might be Googling this info. The following works for me for the 31s1p Xeon Phi:
MB: Asus H97-Plus (So far this is the cheapest board I've found).
CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Ram: 16gb DDR3-LV ram
OS: CentOs 7, kernel: 3.10.0-327.13.1.el7.x86_64
MPSS: 3.7-1 compiled using rpmbuild

MicCheck 3.7-r1

Copyright (c) 2016, Intel Corporation.

Executing default tests for host

Test 0: Check number of devices the OS sees in the system ... pass

Test 1: Check mic driver is loaded ... pass

Test 2: Check number of devices driver sees in the system ... pass

Test 3: Check mpssd daemon is running ... pass

Executing default tests for device: 0

Test 4 (mic0): Check device is in online state and its postcode is FF ... pass

Test 5 (mic0): Check ras daemon is available in device ... pass

Test 6 (mic0): Check running flash version is correct ... pass

Test 7 (mic0): Check running SMC firmware version is correct ... pass

Status: OK

Posted on 2016-04-20 03:21:25
hlide

Thanks, I've bought an INTEL Xeon Phi 31S1P (passive SKU) for my Asus Z170-A without checking the exact requirements and was starting being discomfited. But reading that your MB was compliant, I started to google if "Z170-A" has "Above 4GB encoding" in BIOS and yes it has in BOOT menu! Now, I think the issue is about how to cool my INTEL Xeon Phi 31S1P properly because after some seconds, its blue light is blinking ad eternam and the card is too hot.

My question now: what did you do to have a success after your first post was stating about a failure ?

Posted on 2016-04-23 23:58:30
Wacky Tobaccy

The "Above 4GB" encoding is the first major hurdle. After that, you need a cpu that supports Vt-d. I originally had a Pentium G series, which did not; so I swapped it with an i7. You'll need to confirm whatever cpu you use that it supports that. In the bios, you'll need to go into something called 'system agent' or something 'agent' to enable Vt-d. For cooling, I bought a .8amp blower fan and also a cool master jet flo 120. The 120 is angled underneath the card to blow air on the outside of the heatsink. The blower fan is attached to the back of the card, blowing air out. Just for kicks I also attached an aluminum block to the top metal of the card. Everything is attached with zip ties attached to fans, the card, and the case. After trying CentOs 6.5, 6.7 and Fedora 24, I was finally able to get it working with CentOs 7 using v7-1 of the drivers (compiled via rpmrebuild). Good luck! :)

Posted on 2016-04-24 11:59:20
hlide

Thanks! I have "Above 4GB encoding" and "Vt-d" enabled and was at least able to boot to Windows 10 with those options so I get more hopes for the rest. Now, I must install an SDK and maybe update the firmware. As for the cooling, I still need to think about it.

Posted on 2016-04-25 11:45:37
Wacky Tobaccy

Don't power up the card too long without cooling if you don't want to see it burn up...

Posted on 2016-04-26 05:37:18
StevenNYC

Thank you very much for the info. I will try to use your hardware config with win 10. :)

Posted on 2016-05-02 01:50:30
Zibi Holka

When you enable 'Above 4GB encoding' in bios, save and enter again to the BIOS.
Does it keep 'Enabled' ?

When its keep 'Enabled' there is chance that is going to work - plz see my previous posts with custom mpss modules for Linux kernel.

Posted on 2016-07-10 17:33:47
Wacky Tobaccy

Yes, it keeps the 'Enabled' setting. Please see my other posts on how I got it working using CentOs.

Posted on 2016-07-11 00:40:02
ivan viti

Dr. Kinghorn,
I have a 5110p xeon phi coprocessor. On the website, it basically lists a bunch of costly "server boards" as compatible motherboards. I was wondering if there is something special about a server board, or if a motherboard that can support a 3100 series card can also support a 5100 series card.

Ivan

Posted on 2016-07-08 17:34:52
Donald Kinghorn

Hi Ivan, I just saw your comment ... Any board that will support the 3100 series will work with your 5110p. You will have to come up with a good push-pull cooling solution for that passive card though. Without air moving through those things they will fry pretty quickly.

A lot has changed since I first wrote this post. It is now common for even consumer motherboards to have lage BAR support ( above 4G decoding). You are pretty safe with any Supermicro board. Others will likely be OK too. (You should always look through a BIOS manual for a board before you buy it)

Best of luck with your Phi they are very interesting cards! ... be sure to get some good airflow through that passive card!
Best wishes --Don

Posted on 2016-07-13 17:54:15
Leandro Mengue

Hello Guys,

I need a urgent information: do you know if the motherboard Intel S2600CP2 works with the Xeon Phi 5110P ?

Best regards,

Leandro Mengue
Brazil

Posted on 2016-09-06 14:00:15
Donald Kinghorn

look at this doc ... seems it may work but will bios update ... read the info for some board quirks ...Cheers
http://www.intel.com/conten...

Posted on 2016-09-08 01:35:12
AnthonyKavassis

Dear Dr Kinghorn although it has been some time since this thread was posted, I thought it would potentially be useful to someone to note that the ASUS Sabertooth Mark 2 USB 3.1 is compatible with Xeon Phi 3100A card. I have had the card running under Windows 10 on this motherboard. One thing to note that it is necessary to tweak the memory remap setting in order for the correct IRQ to be set for the Xeon Phi card. The Sabertooth motherboard is in fact a socket 1150 which makes it a more economical proposal as I have a Xeon E3-v1246 running on this system.

Posted on 2016-12-14 18:59:15
dkinghorn

Hi Anthony, even though this post is quite old it is still a valuable resource since there lot of those older Phi cards available. They are still very interesting devices and great to try out parallelization and vectorization ides with! Thanks for posting the info on the ASUS Sabertooth and the tip on IRQ you may save someone a big headache at some point.
Best wishes --Don

Posted on 2016-12-16 21:07:22
AnthonyKavassis

Hi Don although I have not had the opportunity to obtain any of the other Z97 motherboards from ASUS I did note going through all the manuals that for example the Z97-Pro Gamer and Z97-A also have the "above 4G decode" mentioned in the manual. I would assume that like my case, those motherboards that support this BIOS feature should also be compatible.

Posted on 2016-12-16 21:15:12
dkinghorn

Yes, in general it is becoming more common for motherboard manufactures to have "Above 4G decoding" available in the BIOS. There are times when it's needed for high-end video cards these days too. ...

Posted on 2016-12-16 21:47:26
gregge

If someone wrote a HEVC video encoding program for the Xeon Phi board, how well might it perform?

Posted on 2017-01-26 07:25:47
Miguel

Hey guys. Love your stuff. Can you help me out? You know of any skt 1150 or 1151 mobo that will let me use the phi 7120a i got my hands on? Thanks in advance!

Posted on 2017-07-04 23:52:46
Donald Kinghorn

Hi Miguel, Your best bet is probably a Supermicro 1151 board. They have been good about having large bar support (Above 4G Decoding). You will need to look at the MB manual for anything you find and verify that the BIOS setting exists.
Best wishes -Don

Posted on 2017-07-05 18:18:15
Miguel

i'll look into that, thanks. was hoping i could get the job done with a low/mid-price-range mobo.

Posted on 2017-07-05 18:34:22
Miguel

"Above 4GB MMIO BIOS Assignment
Select Enable for remapping of BIOS above 4GB. The options are Enabled
and Disabled." their cheapest 1151 model has that on the manual. would that suffice?

thanks again!

Posted on 2017-07-05 18:41:47
Donald Kinghorn

should be OK, they made a commitment to have it on all of their boards which is why I suggested them ... but it's always good to double check!
Take care --Don

Posted on 2017-07-07 23:11:29
Miguel

Thanks alot!

Posted on 2017-07-07 23:47:04
Ricardo Cancino

hellow I might be a little late, but I have a amd 1950x on a msi x399 pro carbon motherboard. Will the coprocessor work with this configuration? and if it works will the coprocessor do the the same functions of the 1950x?

Posted on 2017-11-01 21:20:42
pavcnik88

I have a question about Xeon Phi 5110P. Will it work on threadripper motherboard? The MOBO is https://www.msi.com/Motherb... and in manual it says that it supports above 4G. Any idea if AMD supports phis?

Posted on 2017-11-19 10:47:26