UCLK: technical limitations or useful marketing tool?

[Massman]

First of all, I'm terribly sorry if this isn't the appropriate subforum for this question, so feel free to move it. Also, I have posted these 'questions', or musings if you will, already on my local forum boards, but the response is non-existant.

Excerpt from forum post:

"This (133x4=666MHz) is the memory running at highest possible frequency, at a stock BLCK frequency. But, that's not the most interesting part about this screenshot. Please have a look at the frequency of the uncore: 1995MHz. On a triple channel i7 platform, the minimum frequency of the uncore would've been 2660MHz, as in 2x the memory frequency (1330MHz DDR); on this dual channel platform, we see that the minimum uncore frequency is set at 1,5x the memory frequency. Why? I don't know. At least, not yet ...

After exchanging ideas with several more knowledgable people, it seems to me that the limitation of a minimum uncore frequency being twice the memory frequency is more of marketing purposes than really a technical limitation. To explain this, let's have a look at the AMD platform first, because it's also a dual channel DDR3 platform. AMD representatives acknowledge the fact that to really put the memory frequency to good use, you need a NB frequency of at least 3x (DDR -> 1,5x) the memory frequency. So, with an IMC frequency of 2GHz, you would only need a memory frequency of 667MHz (DDR3-1333). Anything higher would still scale but less and less intensively, at which you can ask yourself the question if you want to spend the extra money on high-rated memory kits. This 'theory' has been developped, tested and confirmed by Tony of OCZ, click for more information.

For Intel-based platforms, the story isn't that much different: the memory controller is now integrated in the processor and the clock frequency of it can form a bottleneck with high-frequency memory. The novelty about the i7 was the newly introduced third memory channel, which should increase the memory bandwidth significantly. Many tests, however, confirm that the extra channel doesn't have that much of an effect in most benchmarks, let alone in daily computing activities. And that is a big problem when trying to sell the product: who wants to pay more for something that doesn't work in the first place? The technique is quite simple: make it look like it works. And that's where the limitation of "uncore >= 2 x memory" kicks in: with an even lower uncore frequency, the added memory channel would have had even less effect than it has now. Less than almost insignificant, that's bad PR. Technically, it seems possible for the uncore to run at a lower ratio than 2:1, but weirdly enough none of the motherboard manufacturers seem to have added this option to their bios, although it would help people reaching 2000CL7 on air cooling since the memory overclock is very often limited by the uncore frequency."
(~ http://www.madshrimps.be/vbulletin/f10/ ... d80-65278/)

The question is in fact quite simple: am I being too critical or thinking too much in lines of conspiracy theories to believe that manipulating the Uncore frequency is just a marketing tool rather than battling with technical limitations? The LGA1156 platform shows me that it's perfectly possible to have an Uncore multiplier running lower than 2x the memory frequency and I'm quite reluctant to believe it's because of the missing third memory channel.

Also, it seems that the i5 7xx series only have memory ratios upto 2:10 (or 5x), whereas the i7 8xx series have ratios upto 2x12 (6x) ... to feed the 8 threads which are present on the 8xx, but not on the 7xx? In any case: for more multipliers, you need to pay more. Coincidence or marketing strategy?

In any case, if it's indeed just a marketing tool, and there's no technical limitation regarding Uncore/memory frequency, why has no motherboard manufacturer been trying to figure how to 'crack' the limitation? I mean: most of the performance enthousiasts are ignorant when it comes to finding the right balance between frequency and timings; 90% just applies the "more equals better"-rule and buys $350 2000CL7 memory kits only to find out their CPU isn't capable of running 4GHz uncore on air cooling. Having a motherboard that allows users to downclock the uncore would be a smart move marketing-wise.

[M_S]

First of all, I'm terribly sorry if this isn't the appropriate subforum for this question, so feel free to move it. Also, I have posted these 'questions', or musings if you will, already on my local forum boards, but the response is non-existant.

Not a problem

Excerpt from forum post:

"This (133x4=666MHz) is the memory running at highest possible frequency, at a stock BLCK frequency. But, that's not the most interesting part about this screenshot. Please have a look at the frequency of the uncore: 1995MHz. On a triple channel i7 platform, the minimum frequency of the uncore would've been 2660MHz, as in 2x the memory frequency (1330MHz DDR); on this dual channel platform, we see that the minimum uncore frequency is set at 1,5x the memory frequency. Why? I don't know. At least, not yet ...

That one is easy- at least with a few assumptions. In triple channel mode, the uncore clock has to be twice that of the memory because in a triple channel configuration, the combined bit bus width meets the interface between the L3/uncore to the CPU that we assume to be at least 96 bit wide (1/2 of the 192 bit wide memory data path). In other words, every triple-channel memory transaction has to be split into two transactions between the uncore and the core, or else the L3 if it is used for prefetch. Only caveat is: we don't know the width of the uncore-core interface on the Core i7 (central queue) and I have not been able to find any conclusive data on this feature. However, if the uncore (or non-core in Intel's parlance) is interconnected with the core through the assumed 96 bit interface, then a 128 bit memory transaction will take at least 1.33 cycles to transfer. If you make the uncore clock 1.33 x of the memory then you use the entire possible bandwidth of all registers that are involved but you will end up with alignment issues in that the first transaction ends 1/3 into the register, the second one will have to start overlapping at 1/3 and end at 2/3 and so on, making things a bit complicated. Easier is to throw away a bit of frequency and use a 1.5 x clock where you have always full transactions with a boundary at 50% of the register width. You throw away a few bits but management is much easier that way.

Off all the different features on the architecture, that particular interface is one of the less likely items to be changed, whereas the memory controllers are just modular blocks that can be thrown in or deleted at lib.

After exchanging ideas with several more knowledgable people, it seems to me that the limitation of a minimum uncore frequency being twice the memory frequency is more of marketing purposes than really a technical limitation. To explain this, let's have a look at the AMD platform first, because it's also a dual channel DDR3 platform. AMD representatives acknowledge the fact that to really put the memory frequency to good use, you need a NB frequency of at least 3x (DDR -> 1,5x) the memory frequency. So, with an IMC frequency of 2GHz, you would only need a memory frequency of 667MHz (DDR3-1333). Anything higher would still scale but less and less intensively, at which you can ask yourself the question if you want to spend the extra money on high-rated memory kits. This 'theory' has been developped, tested and confirmed by Tony of OCZ, click for more information.

Different architectures and interconnect width will result in different requirements for the interaction of the different components. The AMD interconnect, for all I know, is totally different from the Intel architecture, there you have independent memory controllers, whereas, in Intel's approach all three controllers are always doing the same thing (even if the physical addresses may possibly vary. BTW, we found the same thing with respect to memory frequency vs. NB frequency and resulting performance scaling, 2.4 GHz is the minimum to get DDR3 1600 really going.

For Intel-based platforms, the story isn't that much different: the memory controller is now integrated in the processor and the clock frequency of it can form a bottleneck with high-frequency memory. The novelty about the i7 was the newly introduced third memory channel, which should increase the memory bandwidth significantly. Many tests, however, confirm that the extra channel doesn't have that much of an effect in most benchmarks, let alone in daily computing activities. And that is a big problem when trying to sell the product: who wants to pay more for something that doesn't work in the first place? The technique is quite simple: make it look like it works. And that's where the limitation of "uncore >= 2 x memory" kicks in: with an even lower uncore frequency, the added memory channel would have had even less effect than it has now. Less than almost insignificant, that's bad PR. Technically, it seems possible for the uncore to run at a lower ratio than 2:1, but weirdly enough none of the motherboard manufacturers seem to have added this option to their bios, although it would help people reaching 2000CL7 on air cooling since the memory overclock is very often limited by the uncore frequency."[/i]
(~ http://www.madshrimps.be/vbulletin/f10/ ... d80-65278/)

You are mixing two different things here. One is the synthetic throughput that really depends on the uncore running 2 x the frequency of the memory, the other one being the fact that finally the actual core is starting to saturate with the actual amount of data that is incoming and that the processing units cannot digest the data as fast as they are delivered. That is one main difference to the older Intel architectures including the Core2 where memory bottlenecks were the biggest problem (even though it wasn't really the memory but the AGTL bus and the fact that every request had to be snooped on a bi-directional bus, leading to some 50-70 % of possible memory utilization only.

The question is in fact quite simple: am I being too critical or thinking too much in lines of conspiracy theories to believe that manipulating the Uncore frequency is just a marketing tool rather than battling with technical limitations? The LGA1156 platform shows me that it's perfectly possible to have an Uncore multiplier running lower than 2x the memory frequency and I'm quite reluctant to believe it's because of the missing third memory channel.

Actually, if you do the math, then that's what it comes down to. Intel has vast experience with this type of data buffering from the days of the AGP bus (internally or externally), I have done enough reverse engineering on this feature (which was originally developed by HP IIRC). A 2:1 ratio is the easiest because you don't have to do any split-transactions, instead, you transfer the entire width of the register every time. With a 1.5 x ratio it is still easy because you split the register down the middle, so you don't run into alignment issues that you would have with a 1.33 x multiplier where you would be stuck with 3 segments and have to track where the boundary ends up after each transaction and subsequent re-fill. This is actually the tidbit that makes me believe that the interconnect is less than 128 bit wide, otherwise the dual channel memory could be transferred in a single transaction but again, most of what I wrote is based on that one assumption of a limited uncore-core bus-width.

Also, it seems that the i5 7xx series only have memory ratios upto 2:10 (or 5x), whereas the i7 8xx series have ratios upto 2x12 (6x) ... to feed the 8 threads which are present on the 8xx, but not on the 7xx? In any case: for more multipliers, you need to pay more. Coincidence or marketing strategy?

Those ratios will change with the actual release of the parts into the market.

In any case, if it's indeed just a marketing tool, and there's no technical limitation regarding Uncore/memory frequency, why has no motherboard manufacturer been trying to figure how to 'crack' the limitation? I mean: most of the performance enthousiasts are ignorant when it comes to finding the right balance between frequency and timings; 90% just applies the "more equals better"-rule and buys $350 2000CL7 memory kits only to find out their CPU isn't capable of running 4GHz uncore on air cooling. Having a motherboard that allows users to downclock the uncore would be a smart move marketing-wise.

Because if they did, the processor would run into a buffer overflow. Imagine you have one register of 1/2 the bus width and then you force it to cycle at less than 2 x the speed. That is you have one big mug of coffee that you are trying to drink but the mug is too heavy so you have a small cup as intermediate carrier. The big mug has twice the volume of the small cup. How many times do you have to empty the small cup if the big cup fills up once every minute?

I'll check on the interconnect width again, I might be wrong but I think I remember having this discussion with some Intel folks.

[M_S]

BTW. the bus width mentioned is just data and does not include ECC bits and so on. I did find a reference in one of the Intel Nehalem data sheets referring to the Qclk (queue clock) as hard coded to be twice that of the memory clock, which leads me to believe that this is a technical necessity indeed, rather than some marketing spiel.

[KTE]

Good points there MS.

We had discussed this earlier about there being internal bus width limitations in Nehalems memory interface which throughputs the data to the Core region...

I think what is being asked is to clarify if there really is a physical limitation or a technical limitation imposed as a physical limitation by Intel's choice.

Even though Lynnfield is a different die which allows setup differences, it can't have a different Uncore physical limitation since the architecture guide where this limitation is stated belongs to the Nehalem architecture rather than any particular core. Lynnfield falls into that same architecture.

So to go on further in presumptions; how exactly can the internal IMC width be different from Lynnfield to Bloomfield? That requires an architectural change of the whole IMC block and configuration. Just as MS mentioned, they can't use odd Uncore speeds with triple channel B/W because it drops the efficiency of the throughput down and potentially causes mismatch. Maybe even causes processor stalls.

Is this really a physical limitation or a technical limitation? It looks like the latter to me + marketing.

3 channels = 2x Uncore limit a) to make sure the Core is never B/W starved b) is the fastest in CPU/RAM performance
2 channels = lower Uncore limit to make sure a) Uncore/CPU/RAM is slower performance than the top-end product b) to allow lower voltages, power and higher binning margins for a lower priced CPU c) to provide it Uncore speed sufficient for 2 channel B/W.

So I think there's an element of both in there. Every business wants you to buy it's fastest and most expensive gear in the highest volumes. They do their best to try and make you. Intel need to differentiate their product and they will certainly impose some limitations so that it doesn't enroach on the higher brands performance segment AKA sales. Usually, as we saw in Kentsfield and Yorkfield, that also means lower absolute overclocking speeds and lower overclocking per voltage for the cheaper product line.

I'm predicting Intel will try to favor Core i7 sales as much as possible and if they think things aren't moving ahead fast enough, they'll play the Q6600 card. Mid-priced but increase its overclocking margins to near the higher-end models so it becomes a hit.

As to the memory multipliers - that is highly likely a BIOS level limitation. Whether by Intel or by the MB MFG, I don't know, and you do have MS saying that will change upon release.

Thanks for the data and posting it here Massman, I hadn't seen it. Also congratulations on your successful inroads with the subzero and super-subzero fields. I know it's what you were trying to break into, good to see you in there.

One thing I couldn't help myself chuckling to is... MSI's board looks very good, one of the best in overall balance, but look at how MSI named it - GD80, AKA better and higher up than GD70, the AMD AM3 counterpart. Now that's what you call marketing.

[M_S]

Actually, it'll be much easier to find out what is going on as soon as Westmere is released because the memory controller is on the auxiliary "uncore", that is, on the die also featuring the graphics controller whereas the core itself is just going to be exactly that. This way it is going to be much easier to figure out the interconnect width, my guess is that it is a full duplex connection anywhere between 96 and 128 bit width. Bear in mind that with these massive parallel buses, any clock skew at that speed will become absolutely fatal and that is one of the reasons to keep the interconnects as narrow as possible unless you are going to arrays of serial interconnects like in the case of the QPI interface but adding differential signaling pairs would be pretty huge in terms of cost etc. The L1 / L2 interconnects are 256 bit wide (IIRC) and that is already pushing things at that speed even though the stubs are extremely short.

[KTE]

Yeah, Core 2, K10 and Nehalem all run a 256 bit L1/L2 bus albeit with differences.

If they're moving Uncore to the IGP die in Westmere, does that mean all the major components currently situated in the Uncore (like the PLLs, MCs, L3 and PCU) are also being moved?

[Massman]

First of all, thanks for the lengthy post MS!

That one is easy- at least with a few assumptions. In triple channel mode, the uncore clock has to be twice that of the memory because in a triple channel configuration, the combined bit bus width meets the interface between the L3/uncore to the CPU that we assume to be at least 96 bit wide (1/2 of the 192 bit wide memory data path). In other words, every triple-channel memory transaction has to be split into two transactions between the uncore and the core, or else the L3 if it is used for prefetch. Only caveat is: we don't know the width of the uncore-core interface on the Core i7 (central queue) and I have not been able to find any conclusive data on this feature. However, if the uncore (or non-core in Intel's parlance) is interconnected with the core through the assumed 96 bit interface, then a 128 bit memory transaction will take at least 1.33 cycles to transfer. If you make the uncore clock 1.33 x of the memory then you use the entire possible bandwidth of all registers that are involved but you will end up with alignment issues in that the first transaction ends 1/3 into the register, the second one will have to start overlapping at 1/3 and end at 2/3 and so on, making things a bit complicated. Easier is to throw away a bit of frequency and use a 1.5 x clock where you have always full transactions with a boundary at 50% of the register width. You throw away a few bits but management is much easier that way.

Off all the different features on the architecture, that particular interface is one of the less likely items to be changed, whereas the memory controllers are just modular blocks that can be thrown in or deleted at lib.

Okay, so if I understand this correctly we have the following situation:

1) The bus width between core and uncore is 96 bit; value is well-founded assumption
2) The triple channel memory data path width is 192 bit
3) The dual channel memory data path width is 128 bit.

So, in order to complete a full 192 bit memory transaction it has to be split up into two because of the small(er) bus width between core and uncore. 192/96 = 2. Thus, in order to make this transaction more efficient (faster), the uncore clock frequency is set to 2 x the memory frequency so that for every memory clock cycle there are two uncore clock cycles. In this case, the full 192 bit memory transaction can be completed in one memory clock cycle as the uncore is now capable of handling 2 x 96 bit in that one cycle.

The problem with dual channel is now that the ratio between memory data path width and core-uncore bus width is 4/3, which makes it more difficult to optimise the memory transaction process. At first sight, increasing the uncore frequency to 1.33x would make it possible to transfer the complete 128 bit, but you'd only use 1/3 of the available 96 bit bus (second clock cycle). In itself not a problem, but you'd have to allign the 1/3 and 2/3 mark in the register. To make it easier you can increase the uncore frequency to 1.5x, so the only alignment needed is at the 1/2 mark.

So, Intel could've gone for the 1.33x as well as the 1.5x, but 1.5x just makes things much easier? It's not really a physical limitation but rather an elegant solution to a technical problem. Just optimising the platform.

Question: Would it technically be possible to run the uncore frequency 1:1 with the memory frequency? I presume it would be, but it would induce a lot of problems as half of the 192 bit data path would have to be refreshed.

(Later on the reply, I noticed you actually have already answered this question )

Answer: Because if they did, the processor would run into a buffer overflow. Imagine you have one register of 1/2 the bus width and then you force it to cycle at less than 2 x the speed. That is you have one big mug of coffee that you are trying to drink but the mug is too heavy so you have a small cup as intermediate carrier. The big mug has twice the volume of the small cup. How many times do you have to empty the small cup if the big cup fills up once every minute?

I noticed that Intel actually recommends running the uncore at "2x+1" of the memory frequency in triple channel configurations. Any idea why? Also, in the more extreme overclocking areas of the IT community some people state that an uncore frequency higher than 5GHz (let's say 5.2) would be less performant than a tad under that mark (4.7-4.8) due to 'internal latencies'. Any ideas on that subject? I'd do the tests myself, but at the moment I lack the right silicon to do so .

Different architectures and interconnect width will result in different requirements for the interaction of the different components. The AMD interconnect, for all I know, is totally different from the Intel architecture, there you have independent memory controllers, whereas, in Intel's approach all three controllers are always doing the same thing (even if the physical addresses may possibly vary.

I see.

Would that be one of the reasons why on an AMD platform it's no problem to use a the CPU frequency well below memory frequency, whereas on an Intel i7 platform that seems to be impossible. Whenever I try to run CPU < memory frequency (ddr rate) the system just refuses to boot.

I'm predicting Intel will try to favor Core i7 sales as much as possible and if they think things aren't moving ahead fast enough, they'll play the Q6600 card. Mid-priced but increase its overclocking margins to near the higher-end models so it becomes a hit.

At the MSI overclocking finals in Munich there was a presentation by one of the Intel engineers and he was indeed pushing the i9 as the most interesting platform for performance enthousiasts. In my opinion, with a little bit of overclocking you can outperform any Core i7 LGA1366 on the market with an LGA1156-based system. The extra's coming from that third memory channel are just not thát significant.

If they're moving Uncore to the IGP die in Westmere, does that mean all the major components currently situated in the Uncore (like the PLLs, MCs, L3 and PCU) are also being moved?

As far as I know, the IGP is being placed on the uncore die (on which the memory controller is placed as well). Nothing has to move.

[Massman]

//EDIT: Seems like the pictures are a tad too big.

I put together a couple of charts regarding the performance scaling of CPU/MEM/UNC using Lavalys Everest and Pifast (don't shoot me , just wanted to keep the testing relatively short). Since Core i7 pretty much stands for 'multiplier-overclocking' I used simple values to test scaling:



Here are the results of the 'doubling in frequency'



6/12 stands for: 6x memory, 12x uncore ratio. Same rule for the two others.



12/24 stands for 12x cpu, 24x uncore ratio.



12/6 stands for: 12x cpu, 6x memory ratio.

Haven't really had the time to fully analyse the charts, though. Above charts are the direct performance scaling results; afterwards, I also charted the indirect results (basicly comparing the effect of doubling a second frequency; e.g.: double UNC freq when CPU freq has been doubled).



CPU -> UNC = When the CPU frequency has been doubled, what is the effect of doubling the uncore frequency as well.

This chart is basicly a representation of the previous three ones.

To end with, I also compared the platform scaling (= increasing the CPU/MEM/UNC frequency at once). First row is the percentual increase going from (cpu/mem/unc) 12/6/12 to 24/6/24 and the second row is going from 133 to 167MHz BCLK (4G/1G/4G). No chart, but it's quite easy to grasp the significance of the raw data.



Now, the interesting part would be to see how this changes in dual and single channel configurations. Something for later this week.

[M_S]

I didn't really go through all of the data you posted (sorry) but most of what I see is exactly what I would expect. L3 entirely depends on Uncore frequency and nothing else. Read and Write (main memory) depends on the memory frequency and the core (after all, the core has to be able to digest the data) and what you show is pretty much exactly what we were saying above, the Core i7 finally has reached a level of interacting with the memory that provides a surplus of data (contrary to previous architectures that were all memory starved) so it becomes a matter of the core frequency whether the core can handle the flood of data even in simple applications like stream (Everest is in effect some derivative of the original "stream" developed at U. Virginia many years ago). What is interesting about your data is that they show how carefully balanced the entire Nehalem architecture is, in that any deviation from the default "ratio" either does not result in performance increase or even drops a few points.

Now, granted, memory bandwidth is not the mother of performance but it is one of the prerequisites - the necessary evil of computing as implemented by Intel where the emphasis is on bandwidth rather than on latency - which is somewhat the path taken by AMD.

[KTE]

I think you know the answer to some of your questions.
MCs have internal latencies like all bus links, just as Intel MCH controls to allow FSB MHz. Performance of the connection is a compromise between latency and speed (as well as width). The people who relax the internal MC latencies largely to get high speeds often end up completely fudging the efficiency of the increased speed and then end up making wrong conclusions because of their limited understanding gained by their restricted methods. You reach a point whereby the MC won't gain in speed but at the cost of huge latencies (just like RAM) which ultimately negate it's speed increase in terms of actual performance. There is always a sweetspot for the compromise. Much of the internal latencies are hidden though and automatically set by the BIOS routines.

Tony at XS made such a mistake with Agena last year when he was making IMC performance assumptions against what all the independent testers including the author of this was finding since we had the time to test and post with the same configurations. It's a constant thing online, eP arguments and replies without aiming for the truth in things. What you need is independent people running the tests to confirm and substantiate what others claim, especially others with a lack of knowledge on the topic area (never seen people on XS daily fight over and mass-believe 250MB/s 4K random write of RAID 0 low-end SSDs where they only bench the cache of their PCIe RAID controller?). That also applies with people you're mentioning who are saying there is lowering of performance over a certain IMC speed. Easy test for that is WinRAR; heavily L3 cache and RAM performance dependent. If a logic/cache is clocked higher, stable, and at fixed latencies, it will not perform lower than at a lower speed with an accurate polling.

AMD K10 has a Core:NB speed limitation and a recommendation too. It's stated in the microarchitecture guide and I don't remember it offhand. In effect, MC should gain as much speed as possible without losing drastic efficiency. Top-end boards end up setting loose latencies for the MC allowing higher CPU-NB clocking. AMD has publically stated in Agena times that the CPU-NB would be clocked 1:1 if everything was ideal (like K8). But it's well known (unlike Intel Nehalem) they have faced large issues and limitations with it.