The Street:

Hello Sony, Goodbye Omnivision. Apple has picked Sony’s 8-megapixel camera chip for the 2011 iPhone, according to supply and manufacturing sources, said Rodman Renshaw analyst Ashok Kumar.

As someone who has installed and run various Linux distros on multiple PS3s since it was first available (two launch PS3s; 60GB JPN  & 60GB US), allow me to explain why the loss of Linux on the PS3 isn’t much of a loss at all.  Linux on the PS3 did serve a unique purpose for individuals interested in IBM Cell development, and organizations that have used the PS3 in clusters (such as the military), but those machines are dedicated to a specific task and are not used to play games or log onto PSN (and won’t be updating firmware anyways).  The real people that this firmware update will impact are casual Linux users that play games, log onto PSN, as well as use Linux.  The problem is, Linux on the PS3 was never really that useful to begin with, and people that have actually used Linux on the PS3 (like myself), are very few, and very far between.  And many of those people have come to the conclusion that Linux on the PS3 has very little utility and isn’t really more then a technical curiosity.

The biggest issue with Linux on the PS3 is that you only have access to 256MB of RAM.  But in practice you really have much less then 256MBs, due to various overhead you actually only have closer to a paltry 196MBs of RAM that is actually accessible by the system.  Consider that the applications and everything else needs to fit in this 196MB space.  Companies such as Yellow Dog Linux have made a business of making a stripped down Linux distro specifically for the PS3 that conserves those limited resources.  Even in 2006, when the PS3 was first released this was considered small, in the modern context, its plain ridiculous.

The second problem is that ‘Other OS’ doesn’t have drivers for the RSX, which means the graphics are largely handled solely by the PPE core of the Cell processor.  Combined with the anemic availability of RAM it means that the default resolution for many PS3 Linux distros is an odd 1124×644 resolution.  There are easy ways of adjusting this, but due to its limitations the PS3 has difficulty handling higher resolutions very well, much less anything relatively graphically complicated without dedicated SPE programming.

The third real issue is that you need to find PowerPC (PPC) stable builds of applications you want to use.  PPC has not been aggressively maintained since Apple moved to x86 for OSX.  This means that a lot of crucial programs, such as Flash, plain don’t work.  The current workaround to get Flash functioning on the PS3 is through Gnash, which is far from ideal, and isn’t completely stable or compatible in all situations.  Linux running on a x86 machine does not have these problems.  For the casual user not interested in the Cell processor, any cheap Atom-based netbook or an old desktop would functionally be superior in running Linux to the PS3 both in terms of compatibility as well as overall performance.

Linux on the PS3 was really meant for IBM Cell development, and for that niche it functioned wonderfully.  There are numerous examples of PS3 clusters that perform admirably for the time.  However, now that general processors are incredibly fast, and the emergence of affordable GPGPUs, the PS3 for use in clusters has their days numbered.

By principle, its understandable to be angry at Sony for a loss of functionality, however, that anger should also equally be directed at George Hotz, whose hack has prompted the removal of ‘Other OS’.  After all, the PS3 was the only console to have officially supported Linux out of the box, and they have kept it in the PS3s only until it was exploited.  There was a degree of good faith to put this seemingly ancillary feature into the PS3 in the first place.

The bigger issue for the average PS3 user isn’t where to put the blame, there is a serious risk that a completely hacked system would mean that trophy data could also be altered; the PSP doesn’t have trophies for this very reason.  The authenticity of any trophy could come into question.  Worse, it does open the possibility for cheats and hacks for competitive online games to be easier to implement as well.   I few people with a hacked console could negatively impact the gaming experience of other players online.

Outside of a limited niche of Cell development, Linux on the PS3 has been close to a novelty for all but a limited few.  As a loss of feature it won’t be missed.  But the sadder reality is that even while the implementation of Linux on the PS3 was indeed shoddy, it did give full access to another OS and the Cell itself.  There have been some interesting results by the commercial industry with the PS3 Linux, but the community in general was always small and has been diminishing gradually.  “Homebrewing” really never took off on the PS3 outside of some sporadic examples, even for a system that was open from day one, perhaps its because of the limitations I’ve presented here, but also perhaps because ‘homebrewing’ needs the attraction of more nefarious intentions as well.

In term of innovation in gaming, Nintendo itself is a very aggressive company; from the design of the original d-pad on the NES/Famicom, to the analog-stick in the N64, to the touch-screen in the DS, to the motion controls in the Wii.  Nintendo has always been a company that experiments with new ways of interacting with technology, they have also had several lesser known, lesser successful, endeavors like the Famicom Disk System, Virtual Boy, 64DD, and the Satellaview that brought online gaming in the mid-90s.

However, Nintendo has not been a company that develops the key enabling technologies from scratch, rather they are a company that uses existing technologies and reinterprets them for entertainment purposes.  For this reason we can examine existing technologies and analyze how Nintendo may be adding 3D display technology to the 3DS.

Autostereoscopic Displays

The technology to display 3D without glasses has existed for close to a decade.  Sharp first released a cellphone in 2002 called the SH25LiS for NTT DoCoMo in Japan that came with a 3D display, this was one of the first models with a display switchable between 2D and 3D modes.  In 2004, Sharp announced a 15″ 3D monitor called the LL-151D, and they have has also released laptops with 3D displays as well.  Phillips until 2009 even sold a 42″ WoWvx TV.  In 2009, Hitachi released the Wooo H001 cellphone that also had autostereoscopic technology, and most recently Fuji has started selling a 3D camera and picture frame both of which have 3D displays.  However, the technology in its current form has its nuances and limitations that better suit smaller screened devices.

To understand these limitations we first need to understand how this technology works.  The visual ability to perceive the world in three-dimensions comes from the fact that each our eyes are receiving two slightly different images; this is called stereopsis.  The technical challenge is to insure that each eyes sees different images from the same display.  With 3D glasses, we can filter the image directly before it reaches the eye, however without glasses this presents a challenge.

There are two primary methods that this is accomplished.  The first method is through a lenticular lens screens, which has small micro-lenses that are put above each pixel, these lenses direct light from alternating pixels to different eyes.  The Phillips WoWvx uses this lenticular lens technology, and most recently, Sumimoto-3M, has also shown a lenticular lens display prototypes that utilize this method.

Sumimoto 3M Lenticular Lens 3D display

The second widely used method is the use of a parallax barrier that is placed in front of the LCD screen.  The images for the left and right eye are placed on alternating pixels; the parallax barrier acts to block out the light from the alternating pixel so that only the corresponding image for each eye is shown to the viewer.  The parallax barrier itself is merely a transparent monochromatic LCD that is placed above the main LCD display.  For this reason, parallax barrier itself can be completely turned off to display a conventional ’2D’ screen.  Sharp, Hitachi, and Fuji use this method for their 3D screens.

The biggest issue with either of these methods is that the 3D image can only be effectively viewed within a narrow spatial window.  For this reason, the viewer needs to be a set distance from display to properly get the 3D effect.  This is also the reason why autostereoscopic TV sets have not been pursued as aggressively for 3DTV sets; multiple viewers and variable viewing distance can quickly ruin the 3D effect.  Also, the larger the screen, the more effort is necessary to keep your eyes within that 3D sweet-spot, and prolonged viewing can quickly lead to fatigue.

For handheld devices, the viewing distance is a lot more predictable; the viewer’s eyes would be between a comfortable viewing distance to no further then an arms length away.  Even for a person with very long arms the actual viewing distance would be fairly consistent.  Furthermore, a handheld gaming device is much more personal, the concern about multiple viewers is also reduced, and the getting yourself in the 3D sweet-spot should be less of a problem with a portable device compared to a fixed monitor or TV set.

Faux-3D.

Another option is giving the user the illusion of 3D while using a conventional 2D screen.  As was mentioned on Kotaku, as well as here, there are various methods that would give the player the illusion of 3D.  The “iPhone method” is through the use of an accelerometer, where titling the device would produce a different field of view.  The problem is that the field of view is based off of the orientation of the device, not the viewer’s eyes.  For that reason the 3D effect is minimal, and accelerometer alone cannot effectively produce a faux-3D in all but the most limited environments.

The more effective faux-3D method is by using eye-tracking technology.  As Kotaku has pointed out, a similar technique is used by DSi game, 3D Hidden Picture, which uses a less sophisticated face-tracking technology (the video below demonstrates the efficacy of a face tracking on a Mario doll).  In that game, the camera is used to adjust the image orientation to hit the viewer’s field of view.  There are also commercial monitors with mounted cameras that use this same principle to produce a 3D-like effect.

Eye-tracking is well suited for portable gaming devices since the user would be positioned less then a meter away from the handheld device and camera.

Nintendo 3DS may use Natal-like technology

If a eye-tracking approach is used a crucial component in this capability would be the camera.  However using a conventional camera to do eye-tracking could be a problem for a portable device.  After all, we should expect that the 3DS would be used in  a large array of lighting environments, on a bus or a train, with constantly variable lighting, at home, on the toilet, or even in complete darkness.  For this reason, a CMOS camera using a normal image sensor cannot be relied upon to eye-track the user in all environmental conditions.

The solution to this, and the prevalent technique which most eye-tracking is done, is through the use of near-infrared TOF (time-of-flight) cameras.  This is the same technology that Microsoft is using to capture full-body motion in the Natal.  The basic principle of operation is that a diode sends out infrared light, and that light bounces off the user and captured by an infrared sensor, and the ‘time-of-flight’ of that infrared light can be calculated.  The main benefit of a TOF camera is that it operates irrespective of visible light.  It can even operate in complete darkness, and mated with a normal camera can cover a large range of environmental conditions.

There are key differences in how the Natal would implement this technology and how the 3DS would.  First and foremost is the fact that 3DS would capture the player less then a meter from the camera, for this reason the infrared diode and camera would be designed to meet these conditions.  Natal, comparatively needs to capture the entire body of the player, and in some cases multiple players.  From that data, Natal creates a 48-point skeletal model of the human player based of that data.  The 3DS would only need to do head-tracking or eye-tracking, and possibly hand-gestures.  Technically speaking, the 3DS would not have the processing requirements that Natal would need if they used a TOF camera.

TOF (Time-of-Flight) cameras can capture images in three-dimensional space that can be utilized in eye-tracking as well as gesture control

What would Nintendo do?

Asahi is reporting that Nintendo will be using Sharp’s displays for their 3DS.  Which means that Nintendo will be using the parallax barrier method to display 3D.  Asahi, we should assume, to be very reliable.  Asahi is a news organization that has a 130 year history in journalism, and are generally considered to be fairly conservative with strict editorial oversight for their business related news.  Sharp, also has a long history of being a supplier to Nintendo; including the DS.  Moreover, Sharp has invested in 3D technology since the early 90s, particular through their European arm.  If Nintendo where to choose a partner for 3D displays, Sharp would be a primary candidate considering their existing relationship with the company as well as their current technical prowess of this technology.

Another thing we should consider is that parallax displays and eye-tracking is not mutually exclusive.  There has been research in mating these two technologies, where eye-tracking could be used to adjust the efficacy of the parallax display.  But as we add technologies we need to consider the cost of these technologies as well.  Nintendo has traditionally shied away from high-cost gaming devices, and instead have relied on the elementary economic principle of supply-and-demand.  From this rationale, we should expect the Nintendo device to be within the ~$200 range, give or take, and we should not be expecting the addition of technology without direct purpose.

As for the parallax display, Sharp has had a difficult time promoting this technology due to the lack of content (its difficult to sell a 3D cellphone when only the menus are in 3D).  Teaming with Nintendo and their developer network gives Sharp dedicated content and a market for their technology.  To add a parallax barrier itself only requires a transparent monochromatic LCD.  What deal that Sharp has with Nintendo is still a mystery, or whether both screens with have 3D functionality.  However, we do know based on the Asahi report that the screen should be below 4″.  Based off of iSuppli, the 3.5″ within the Apple iPhone 3GS costs $19.25.  For this reason we should consider the BOM (bill-of-materials) for the dual LCD screens itself for the Nintendo 3DS to be over $40 without the 3D capability assuming they have a similar resolution.

In term of adding eye-tracking, from a cost perspective, a TOF camera’s CMOS sensor is not necessarily that different from the cameras already on the DSi.  A CMOS/CCD sensor has the ability to capture light in larger range then human’s perception of visible light, and for this reason most camera sensors go through the effort of keeping captured light only in the visible range.  Capturing excessive IR light in normal photography produces fake-colors and is considered an undesirable trait, but there have also been some specialty cameras like the Fujifilm S5 Pro that captures IR, as well as UV, images.  In the context of volume which would be necessary for the 3DS, an addition of a TOF camera would meet sufficient economies of scale, and the cost to implement it should not be significantly more then the two-cameras already present on the DSi, however it will still be an additional cost nonetheless.  Now, if Nintendo will implement parallax display and/or eye-tracking is still speculation.  However adding either or both should still be a significant cost to Nintendo.

Force-Feedback 3D Controls

Nikkei is reporting that the 3DS will have “a 3-D joystick and a force feedback mechanism that will let players feel the collisions of a game character, for example. It had already acquired related patents at the end of last year.“  The Nikkei we should also consider to be a very reliable source, they publish Japan’s primary stock market index, and is generally very accurate on what they report.  This “3D joystick” may only mean a traditional analog-stock, however, the patents Nintendo has acquired last year are of a different nature, so the Nikkei may be referring to something completely different.

Nintendo has traditionally avoided adding graphical enhancements without backing that technology up with a corresponding control scheme.  If we think back to the N64, Nintendo added an analog stick to move characters in three-dimensional space, and added 4 yellow “c” buttons to control the camera angle.  Mario 64 was a revolutionary game for the time, and it was the first true-transition from 2D gameplay into a fully realized 3D gaming experience.  Those “c” buttons has evolved into a second analog stick, but the original concept remains.

When we think of Avatar-like 3D, we expect objects coming out at the viewer in three-dimensional space.  Instinctively, we reach out to touch these visual objects to only have our hands pass through them.  Its an experience I’m sure we have all had when we first saw 3D at the theaters.  So we need to re-investigate the semantics; is Nintendo going to make ’3D controls’ in the vein of what currently exists in the form of analog sticks?  Or is Nintendo going to create ’3D controls’ that exist in three-dimensional space?  Assuming that its the latter there are several technical methods that this could be accomplished.

If Nintendo is going to add a IR TOF camera, the easiest method would be to do finger-tracking.  As the video below demonstrates, a user’s fingers can be made into a 3D mouse and be used to type on a virtual keyboard.  The force-feedback could be given via rumble in the device itself when a corresponding gesture is made.  While Nintendo implementing IR cameras is still speculation, if they are implementing the faux-3D approach via eye-tracking, this control mechanism could be added for little to no additional cost.  Again, much like eye-tracking, finger-tracking can be accomplished without the TOF cameras, however given the variable lighting conditions that the portable would be subjected to, a TOF camera would insure accuracy and consistency.

Another method force-feedback 3D controls could be implemented is through a stylus.  Nintendo has patented related technology, and these patents may be what Nikkei was referring to.  Within this patent, Nintendo refers to a game apparatus that is akin to a stylus, and when used in a gameplay environment, for instance attacking an enemy, vibrations of that gameplay action can be felt directly in your fingertips.  The benefit of using a physical device like a stylus is that it can have buttons on it as well.  If this approach is taken, the overall size and how such an input device would be powered is another question.  It might be a stylus-sized Wii remote, or it could be something completely different.  But there are examples of similar technologies:

Nintendo patent demonstrates force-feedback on a portable gaming device

In the video above, the news reporter is startled as the stylus device he is holding in his hands is mysteriously pulled towards the screen, he laughs as the stylus device is then pushing his hand away from screen.  The scientist explains that there is a small motor within the stylus, and the spinning of that motor in calculated directions can give force-feedback sensations.  The reporter then tests out the pinching and pulling of this 3D ball; he explains that there is a distinct sticky sensation as he moves his stylus.

Conclusion

The technologies presented here may, or may not, be used in the 3DS when its finally showed in June.  What we need to understand is relative to Nintendo’s announcement there is a limited number of options that can accomplish what Nintendo is claiming.  The announcement of the 3DS saw Nintendo stock surge almost 10%, all based on short PDF announcement without much detail.  There are still a lot of unanswered questions, but what we can count on is Nintendo is full of surprises.

What does the iPad, iPhone, iPod, Windows Phone 7, Nintendo DS, Nokia N900, Motorola Droid, Google Nexus One, and the Kindle all have in common?  They are all powered by ARM microprocessors. The ARM architecture has been around since the mid-80s, why is it only now that it has ascended to a level of central importance?  The answer is because we rely on our mobile device not just to make calls, but check Facebook, surf the Internet, watch videos, listen to music, and play games.  ARM has become the CPU of choice for mobile devices, and as the line between mobile and immobile gets further blurred by devices like the iPad and Chrome OS, the ARM architecture has become the most viable x86 killer that has ever existed.

ARM is the democratization of the CPU outside the Wintel paradigm.  Every significant tech company, Samsung, Qualcomm, Cirrus, DEC, Sharp, Texus Instruments, Yamaha, LG, NEC, Nvidia, and even formerly Intel makes ARM processors. Instead of “Intel Inside”, the freedom and openness of the ARM architecture allows Apple to make their own ARM-based SoC (System-on-chip).   Apple A4 CPU in the iPad is an example of an “Apple Inside” situation.  There is nothing to stop a “Nintendo Inside”, “Dell Inside”, “Microsoft Inside”, “Google Inside”, “Qualcomm Inside”, or a “Sony Inside”.  Any company can bypass the Intel paradigm, license the architecture  for themselves, and design and build their own CPU to custom-fit their product.  Just as importantly, the ARM licensing fee has been calculated to cost as little as 6.7 cents per chip.

Most crucially ARM allows fungibility of processors between manufacturers.  For instance, Apple uses Samsung-sourced ARM processors in the iPhone, Qualcomm’s Snapdragon, the CPU used in the Nexus One, has been rumored for both the iPhone as well as iPad; ultimately Apple created their own ARM processor called the A4 for the iPad, and presumably will use variants of that chip for future iPhones and iPod devices as well.  The end-user doesn’t know or care, they both function identically, but ARM offers real free market competition between manufacturers and real interchangeability between makers that keep prices down and quality up.

ARM SoC (System-on-chips) offers freedom of choice

The major strength of ARM is that it excels in performance-per-watt and performance/mm2 (die area); this means a smaller chip, less cooling, smaller batteries, smaller overall device, and dramatically reduced cost.  Companies like Nvidia have packaged the entirety of the Tegra chipset in a size no larger then a USB thumb-drive and claims battery-life measured in the days instead of hours.  ST Telecom has even created a new type of  SIM-card with an ARM-processor, Android OS, and flash memory onto an incredibly small package.   Just as importantly, being that ARM is cheaply licensed and flexible, it can be mated with non-ARM components like graphics-acceleration, to make a complete SoC like Nvidia’s Tegra and Apple’s A4.  This is a flexibility that x86 cannot offer, Nvidia have tried building a x86 SoC solutions, including a rumored x86 processor, but its been an endeavor that has ended with failure and a billion dollar anti-trust lawsuit against Intel.

ARM allows any company the freedom to choose what parts come on their SoC.  The graphics can be from Nvidia, or PowerVR, or ARM’s Mali, a video decoder could be added to play HD video, and features can be added or deleted as the need fits.  The reason this is possible is because the business model surrounding ARM is agnostic to what goes in, or doesn’t go in, the SoC.  Intel for their part is also creating an ARM-competitor for smartphones and mobile devices called Moorestown, and it too is a SoC and is hoping to be on the market sometime this year.  However, Intel’s business model is about selling chips, their SoC is theirs, uses Intel’s graphics, and uses their technology.  Because of this the hardware is designed for the processor, while in ARM’s case, the processor is designed for the hardware.

The closing performance gap

ARM is a fast, low-cost, power-efficient RISC processor, and its an architecture that has been inherently designed for low-power applications.  Hitherto, ARM chips could not compete against Intel’s x86 chips in terms of raw processing power, but recently that performance gap has narrowed and the market dynamics have changed as the demand for high-end desktop CPUs have contracted in favor of lower-cost, lower-power processors like the Atom.  Hence, both ARM and x86 markets have been on a collision course, and within the next few years its expected that ARM processors will start encroaching on devices that have traditionally been dominated by x86.

With the ARM Cortex-A9 the performance gap with x86 has narrowed even further.  As can be seen by the video below, a 500Mhz ARM Cortex-A9 can out-perform a 1.6Ghz Intel Atom in web browsing.  When you consider that the ARM Cortex-A9 chips are expected to hit 2Ghz and have up to 4-cores, you can begin to comprehend the level of performance these ARM chips will be bringing to the table in the coming years.  Just as importantly, each of these cores is expected to consume only 0.25W of electricity per core, making it suitable for a large range of mobile devices.  The upcoming multi-core OMAP 4 processor from Texas Instruments, which is based on the ARM Cortex-A9, boasts that it will bring 1080p video recording and playback, 20-megapixel imaging, and high-end 3D graphics to the smartphone.  That’s not all, in a couple of years, ARM is already readying its Cortex-A9 successor, codenamed Eagle, by then its expected that ARM performance would have grown to the point where they would start competing with Intel in the server market.

ARM: Cheap as chips

Perhaps the greatest advantage that ARM has is in its economics.  Business is business and gold glistens, and there is no greater incentive for the adoption of these ARM chips then its price.  Recently, iSuppli calculated the cost of the Qualcomm 1Ghz ARM chip within the Google Nexus One to cost a mere $30.50.  Apple, by taking ARM development in-house, is estimated to get their 1Ghz A4 ARM chip for a mere $17.  These are both SoC(system-on-chips), meaning that the cost of the CPU, GPU, controllers, etc is all included within that low price-point.  Again, this demonstrates the democratization of the CPU; if the costs of supplying your chip from an external supplier like Qualcomm, TI, or Samsung is undesirable, then a company like Apple can decide to build their own chip if the economies of scale make sense.

Lower-power variants are already below $10 per chip, as ARM performance shifts forward, we can presume that what we consider currently to be high-performance chips will fall to those prices as well.  There is no bottom for ARM chips, they become as cheap as the market allows, and there are no orchestrated marketing exercises to artificially keep chip prices within their market segment.  ARM as a company only licences their design and IP, as stated before, ARM licensing averages out to only 6.7 cents per chip.   If the market demands gravitate to smaller, cheaper, faster devices, then chip design follows those needs; no longer is the CPU maker like Intel dictating market segments and expecting the industry to form around it.

x86 isn’t going anywhere

The reality of course is that x86 will continue to be a central part of the computing industry for the foreseeable future.  The x86-based desktop, notebooks, and servers will continue to exist and be an crucial tool for individuals involved in content-creation.  The businesses and industries that require x86 machines will not disappear, and on the higher-end Intel’s chips, such as the i7, are unrivaled in performance by any ARM chip.  For serious professionals involved with fields like graphics design, 3D graphics, office productivity tools, and software development; x86 is still the CPU of choice.

The doom-and-gloom surrounding x86 and PC-makers are largely due to the constriction of profitability in the PC market due to budget chips like the Atom and Netbooks.  To the average consumer, the Internet is the ‘killer-app’, and Netbooks/notebooks within the $400-500 range offer functionality ‘good enough’ to meet their needs.  Sony’s VP called this a ‘race to the bottom‘ and Acer founder has predicted US PC-makers to become extinct.  This is where the battleground with ARM will be fought, from the bottom up, with devices like the iPad and operating systems like the Chrome OS.  The PC will continue to survive, and occasionally thrive, within a higher-level niche.

Will we see an ARM-compatible Windows 7 or Mac OSX?

The greatest advantage that x86 has is Windows and Mac OSX.  Particularly in Windows there really is no other option other then x86.  But that may be changing, future ARM support for Windows 7 (or possibly Windows 8) has been hinted at by people such as ARM’s CEO.  Windows CE variants already support ARM, and their mobile platforms such as Windows Phone 7 and their Courier device shows how seriously Microsoft is taking those market segments.

While Apple’s plans are very secretive, their iPhone/iPad OS is an OSX derivative that already runs on ARM, and Apple is now in the ARM chip-making business.  It stands to reason that there is a large incentive for Apple to move to ARM in their future desktop OS, they’ve already demonstrated the willingness to move between architectures with their jump from PowerPC to x86, a future Macbook using an Apple-designed dual-or-quad core 2Ghz ARM Cortex-A9 running on OSX or OS11 is not inconceivable.

Ultimately, a move to ARM would require a hypervisor to maintain a level of compatibility with the x86 variant of the OS.  Google seems to be taking a similar approach of x86 virtualization by using their Native Client (NaCl) for cross-compatibility between ARM and x86 on their upcoming Chrome OS.  These virtualization tools have also matured with contemporary examples such as Intel VT-x and AMD-V, but there still is a substantial technical challenge.  While we shouldn’t realistically expect ARM-variants of either OS in the immediate future, there is a financial incentive to move to lower-cost ARM chips as a method to regain profitability in the struggling PC and desktop OS market.  An ARM-based Windows/Mac opens up the possibility of a Dell PC running a Dell CPU, or Apple OSX using the same processor as in a future-faster iPad.

Mobile and beyond

In one form or another, ARM will play a central role in our digital lives. It may be in the form of a phone, iPad, portable game console, or possibly even a desktop OS.  While the ‘death’ of x86 may be an over-exaggeration, as it currently stands, ARM has changed the way we interact with technology and the way the chip-industry does business.  The ubiquity of ARM can be attributed to its openness and its business model that gives manufacturers, and ultimately consumers, a larger say on the processor that goes into their products.

As first reported by GoRumors, Sony has patented a “Universal Game Console Controller”; as the USPTO patent describes it will be a “universal game controller which can be used to emulate the controllers of popular game consoles, such as, without limitation, the PlayStation made by Sony, a controller made by Nintendo, X-box game controllers made by Microsoft, Amiga CD-32 controllers, Atari Jaguar controllers, Gravis Gamepad controllers, Sega controllers, and Turbographics controllers“.

Judging by the patent application, the controller would be a large LCD touchscreen with two-shoulder buttons on each side, and will be equipped with its own speaker and rumble capability.  The button layout will be displayed on the screen, and the user would press these virtual controls in lieu of real physical buttons.

Most gamers on the iPhone may be familiar with this concept.  The iPhone’s lack of physical controls have lead many developers to implement multi-touch based d-pads, analog sticks, and buttons.  On the same note, many iPhone gamers may also be familiar with how frustrating and inferior touch-screen virtual buttons are.  Touch-screen controls lack the feedback and accuracy of real buttons, and in its current state is no substitute for real physical controls.   Being that the controller is the primary interface between the player and the game, poor controls will lead to a poor user experience.  So its perplexing that a company with extensive gaming experience like Sony would want to introduce such a seemingly inferior gaming-pad.

However if we take into context Sony’s previous patent then things start making more sense.  Sony also filed for a force-feedback touch screen over a year ago, a technology that Sony calls “tactile pixels“.  The concept is that button presses on a touch-screen would give a tactile response of actually pressing a physical button.   The patent acknowledges the shortcomings of touch-screen controls; ”existing touch screens are configured to receive a mechanical input and provide a visible output,” they “are not configured to provide both a visible and a mechanical output.”  To accomplish this Sony has put a grid of bumps underneath the touch-screen, when a button is presented on-screen, the bumps below woud rise up to meet the users touch giving the sensation of a physical button.

But there are still more things we need to consider.  Patent applications themselves don’t necessarily lead to a final product even though they may indicate the research directions that the company is taking.  Secondly, while the patent was published recently it was originally filed in the summer of 2008.  Third, the patent was filed by Sony Electronics, not Sony Computer Entertainment, which is the division involved with the Playstation.  While none of this conclusively points to if this product will, or won’t, actually hit the market, we have to also consider that this touch-screen LCD controller would be prohibitively expensive for most people.

In Suppli’s recent tear-down of the bill-of-materials of the iPhone 3GS.  The 3.5″ 320×480 TFT screen costs $19.25, the capacitive touch-screen assembly cost an additional $16.  So almost $35 is added to the cost of material for just the touch-screen itself, now when you add this new force-feedback technology, battery, the physical enclosure, and the microprocessor to control it all you are looking at “universal controller” that costs over $100.  Suddenly we are talking Steel Battalion-level console controller prices.  There is no business case for such an expensive device when multiple physical controllers would be cheaper and offer better overall gaming experience.

So it would probably be a mistake to interpret this patent at face value.  Sony is already planning on releasing their own Motion Controller to compete against the Wii and Natal, there is no room for another Playstation peripheral.  But even with their limitations, its safe to assume that capacitive touch-screens are going to be ubiquitous on all future handheld-gaming devices.  Logically, removing physical buttons would be an ideal use of space and improve the design of the handheld.  Could Sony be contemplating a PSP2 or PSP phone with only shoulder buttons and no face-buttons and Instead use resistive-touch virtual-buttons that can be displayed on screen?  While these patents may not necessarily indicate a real product, it does illuminate the direction and technology that Sony is using on their future mobile devices.

Noktor 50mm f/0.95

43rumors is reporting that a new upstart, Noktor, is preparing to release a 50mm f0.95 lens on March 1st.  The lens will be a 50mm with an aperture of f/0.95, the closest comparison (as its name suggests) would the Leica Noctilux 50mm f/0.95; which is a $11,000 lens.  Leica lovingly refers to a f/0.95 lens as “breaking the “sound barrier” of lens speed 1.1 has been the Holy Grail of lens design for many years“, and a lens that can “Out Perform the Human Eye” and a “King of the Night“.  On paper at least, this Noktor should become a poor-man’s Noctilux.

43 rumors also points out that these lenses possibly could be based on the Senko/Navitron/Yakumo lens designs fitted with a MicroFourThirds mount.  Those lenses were primarily C-mount lenses designed for broadcast CCD cameras, and have been frequently modified amongst the M-mount community  for use on their Leicas.  However, it should be kept in mind that a 50mm lens on a MicroFourThirds system will have an equivalent focal length of a 100mm on a full-frame due to the 2.0x crop-factor.  Being that MicroFourThirds cameras can do HD video (which Leica cannot) and has been shown to have impressive low-light performance comparable to dSLRs with larger sensors, this Noktor could become a useful tool in a photographer’s arsenal of lenses.

Noktor has also put up two samples viaTwitter:

Sample 1, Sample 2

“f/0.95″, What Does it Mean?

“f/0.95″ is the aperture size.  In general, the larger the aperture, the more light can enter the camera, the shutter speed can be faster, and photographs can be taken at lower light, but it also means that photographs taken at large apertures will have a much more shallow depth-of-field(DOF).  Sometimes this shallow DOF is desirable because it produces a nice “bokeh” effect that isolates the subject.

Most 50mm prime lenses range between f/1.4-f/1.8, but there are some speciality lenses like Canon’s 50mm f/1.2L that offer larger apertures, but they tend to be relatively expensive.  dSLRs like Nikon’s F-mount and Canon’s EF-mount are typically at a disadvantage in producing incredibly large apertures, this is because dSLRs have a much larger focal-flange distance (FFD) to accommodate the typical SLR mirror-box assembly.  So rangefinders, like the Leica M-mount, can make lenses with large apertures because they can mount the lens closer to the light-sensitive medium (film/sensor) because it has a shorter FFD due to the lack of a mirror-box (and hence also lacks autofocus abilities as well as an optical view finder).

Since the camera has entered the digital era, the mirror-box assembly (and its phase-detection based AF) has become less necessary.  Panasonic in particular has demonstrated that contrast-based live-view style cameras can also produce fairly accurate autofocus comparable to dSLRs.  Since last year, MicroFourThirds cameras that do away with the complex mirror-box assembly and short FFD have hit the market.   Many have started to refer to these cameras as “E.V.I.L” cameras, as in “Electronic Viewfinder Interchangeable Lens”.  This means that compact large-aperture cameras along with modern features (HD video, AF/AE, etc)  are possible with the MicroFourThirds system.  This upstart Noktor has seen an obvious niche that the big lens companies have yet to fill.

Alternatives

There are currently  several ways to mount ultra-fast primes onto the MicroFourThirds system.  The Loctilux 50mm f/0.95 for instance can easily be mounted onto any MicroFourThirds body with an M-mount adapter.  There are also several very-fast primes like the Cosina-made Voigtländer Nokton 50mm F1.1 lens which sells for a little over $1,100 and which can be mounted via an VM adapter (sample).  Cosina, it should be noted, also make the superb Carl-Zeiss SLR lenses for Canon and Nikon mounts.  Nikon has also made a 50mm f/1.1 rangefinder lens in 1956, as well as a 50mm f/1.0 Nikkor O-lens in 1962, but those lenses are expensive and very hard to find.  There is also slightly slower, slightly cheaper, alternatives like the Minolta 50/58mm Rokkor f/1.2 or the Canon FD 50mm f/1.2 that can be mounted via adapters.

The lens that shouldn’t forget mention is the Canon 50mm f/0.95 “Dream lens”, which was originally produced for the Canon 7 rangefinder.  However, there are issues with mounting the lens onto a MicroFourThirds body.  The first method is purchasing a lens that has been converted to an M-mount, or converting the lens yourself by sending it to several specialists.  The lens has been popular amongst the Leica community as there are several M-mount Dream lenses floating around, but they tend to cost over $1,000 and can be hard to find.  The other option is to buy the “TV” version of the lens, which Canon produced for CCTVs, that comes with a C-mount adapter.  From either C-mount or M-mount, the lens can be fitted onto a MicroFourThirds body.

This Noktor 50mm f/0.95 should offer something more easily obtainable without the hassles of finding old-lenses on Ebay or fiddling with adapters; it will be a native MicroFourThirds lens.  The big question is how much it will cost and what image quality it will produce.  Will it have the soft ephemeral “dream”-like quality of other f/0.95 lenses?  Stanley Kubrick famously converted a Zeiss 50mm f/0.7 NASA lens to take the candle scenes in the movie “Barry Lyndon”,  modern sensors mated to these ultra-fast primes could produce similar unique results- especially mated to capable video-centric MicroFourThirds cameras like the Panasonic GH1.  This could be a lens for the modern-day Stanley Kubricks…

Canon 50mm f/0.95 "Dream Lens" mounted on a MicroFourThirds Panasonic G1

Christian Lindholm of BusinessWeek investigates how social networks like Facebook has changed how we use the mobile phone today, and argues that “Facebook could redefine communications by issuing its own smartphone“.  Lindholm asserts that social networks, like Facebook, deserves more integration into smartphones then static apps or websites, that a physical embodiment of  Facebook could ‘revolutionize‘ how we communicate.

The FacePhone

Many might argue that a Facebook app might be more then enough.  However, Lindholm poses some interesting features and concepts that a Facebook smartphone could deliver.  Facebook could serve as the ‘web’s best contact book‘, and a ‘Facebook-centric device.. would have the ability to incorporate real-time information about friends at every stage of your interaction with them. The need to send expensive SMS messages would be reduced or eliminated by the ability to send a Facebook message from your phone as easily as you now send an SMS… A Facebook device would open up new possibilities for creativity, too. We know that consumers more than ever carry around devices for creating and sharing content with each other. Instant and seamless uploads of photos and videos would create not just a master phone book but also a master photo and video album. Indeed, a Facebook phone could become your master life recorder—a kind of social archive of your digital life.“

Lindholm does make some very valid points; email, SMS, and Facebook PMs are all redundant in its functionality.  We check and recheck multiple accounts at multiple times throughout the day.  We may have  photos in our home, on Flickr, on Picasa, or Facebook, on our camera, and on our phone.  We have multiple contact lists, from our email, phone, and Facebook.  The question becomes is there a better way of handling all these inefficiencies?

A Facebook-branded smartphone with a unique OS may not necessarily be required, but currently smartphones like Apple’s iPhone are tightly locked-down on basic functionality that competes with Apple’s own.  Facebook integration into the contact book, SMS, photo album, and the phone itself is not allowed. There is no “app for that”.  As Lindholm puts it, ’Such a device would undoubtedly terrify mobile operators, who could witness even more of their network traffic shift from voice and pricey messaging to raw streams of data. Incumbent handset makers could lose, too, as further control over the user experience shifts to an Internet company that stands to profit from dominating the online life of users.’

Conversely, Google has an opportunity to better integrate Android under a smilar social network approach with their newly announced Buzz, but how open will it be to competing social networks like Facebook?  There is still a lot of questions, but the bigger question is what would such a device be called?  ’FaceBookPhone’, ‘FacePhoneBook’, ‘fPhone’, or merely ‘FacePhone’?