OnQ: Delivering Better Web Experiences for Snapdragon S3 Mobile Processors [Sy Choudhury, Director of Product Management, Web Technologies, Oct 10, 2011]
Sy Choudhury, Director of Product Management for Qualcomm, demonstrates how our web optimizations can enhance the overall web browsing and web apps experience on Android for Snapdragon S3 mobile processor-based devices.
Heavy Lifting on the Mobile Web – Put It Where It Belongs [Sy Choudhury, Director of Product Management, Web Technologies, July 7, 2011]
I encourage you to take a close look at the Snapdragon™ mobile processor– its architecture, documentation and tools – as you consider developing mobile Web apps for Android. The image above maps the Snapdragon processor to the heavy lifting you face in delivering a good mobile Web experience to your customers.
Let’s go through them individually:
Transport– DNS lookup, page loads, page reloads, image downloads…the browser never lets the modem have any peace. But the browser – not to mention the user –is more forgiving on the desktop than on a mobile device. The Web transport functions need to work intelligently on mobile devices or the user experience will drown in latency and needless reloads from the network.
Layout – Images are almost 2/3 of the payload of the average Web page. Do you want graphics-rich sites like photo galleries and social networks hogging the CPU to decode images? The browser needs to take one look at them and offload them to dedicated hardware for decoding.
Rendering – Whether you need to compose pages in a frame or stream mobile video smoothly, there are better places to do it than the CPU. The work of drawing page objects on separate layers and merging them efficiently belongs on a graphics processing unit (GPU), and the coming onslaught of mobile videofavors chipsets with a dedicated multimedia engine.
That’s one big advantage to pulling all of this functionality into a single chip. Another advantage is that it makes things easier for everybody. We produce the components more efficiently, manufacturers sacrifice less real estate inside the device, and you keep your eye on just one set of rules for writing to hardware.
Also, as part of Qualcomm’s Web Technologiesinitiative, we’ve been developing and implementing optimizations for all of this functionality. Qualcomm has made them available as updates to Adobe® Flash® Player and Qualcomm Innovation Center, Inc., our wholly owned subsidiary that focuses on mobile open source contributions, has made them available to the community. You can take advantage of them by developing for the Snapdragon processor, because we also include them as part of the software bundle we ship.
Give your end users mobile web applications that are designed to run like native applications.
Get ready for a few realities about developing for the mobile Web:
- Rich content and complex Web applications also rely on hardware resources deep inside the mobile device.
- You need to give your Web users desktop-quality performance on mobile devices before your competitors do.
To make this easier for you, the Web Technologies initiative from Qualcomm Incorporated and Qualcomm Innovation Center, Inc.(QuIC) enables a series of software features and hardware-tuned performance optimizations that give the Web application environment deep reach into the mobile device. The end result–a level of performance from your Web app that users typically expect from native applications and even desktop applications.
- better interactivity with Web apps and pages
- snappier, smoother response to user commands
- the highest quality and resolution multimedia streams
- lower overall power consumption
- Web application functionality and performance on par with native mobile apps
Web Technologies Tools and Resources
Using our runtime software packages, you can begin developing mobile Web apps that perform more like native apps.
Uplinq 2011 Super Session: Is HTML5 the Future of Smartphone Apps?: A Conversation About Web Technologies
Is HTML5 the future of mobile apps? Can web apps ever perform on par with native apps? What do the advances in browser-based experiences bode for mobile operating systems? How can hardware matter in such an abstracted environment? Join Ben Wood, director of research for leading industry analyst firm CCS Insight, as he engages Rob Chandhok, who leads Qualcomm’s software strategy efforts, on these and other questions central to the intersection of web technologies and mobile.
Uplinq 2011 Session: Session: Developing Rich Web Apps for Smartphones
Sy Choudhury, Director of Product Management for Qualcomm, demonstrates Snapdragon’s the in-page web video capability, HD video at 720p in HTML5 and Flash, and full HD video at 1080p for mobile devices
DASH – Toward a Better Mobile Video User Experience [Sy Choudhury, Director of Product Management, Web Technologies, Aug 16, 2011]
Do you like jittery, staccato playback and long buffer times when you watch video on your phone or tablet? Neither do I. Neither does Qualcomm.
Let’s face it, though: the mobile video genie is out of the bottle, and it’s not going back in. Video streaming continues to dominate mobile bandwidth consumption, accounting for 39 percent of data usage in the first half of 2011, according to the H1 2011 Allot MobileTrends Report. Elsewhere, Frost & Sullivan notes that CTIA has called for an additional 800 MHz of bandwidth to cope with the onslaught of mobile video; the U.S. government is trying to provide 500 MHz of that request.
There’s no simple solution that will ensure a good mobile video experience. We’ve identified areas that are ripe for improvement and are working diligently to address them. DASH – Dynamic Adaptive Streaming over HTTP – is an important one. We see DASH as the industry’s best approach to streaming mobile video, while preserving the kind of video experience consumers expect.
What is DASH?
DASH is an open standard that addresses what we think are many of the biggest problems in delivering streaming video:
1. File size– In the old days, Web video was easy. You downloaded a 2- or 4- or 10 MB file to your device and then played it. That wasn’t really streaming, and it wasn’t scalable – imagine streaming high-definition movies that way. So DASH is a standard for chopping video streams into smaller segments.
2. Changing network conditions– To keep a stream of video flowing smoothly, servers need to send these smaller segments when the device can accommodate them. With DASH, the video lives on the server in several different bit-rates – for example, 250 kbit/s low quality, 500 kbit/s medium quality and 1000 kbit/s high quality. And here is the key; the device determines and then commands the server to send the best quality given the current network conditions (see diagram).
3. Proprietary formats– Most Web video is encoded in common codec formats like H.264 but stored in various streaming formats, depending on the media player on the device. Adobe, Apple, Microsoft, Netflix and many of the other names you associate with video delivery have their own streaming format and their own approach to streaming. DASH defines openly published profiles and the device’s native player can therefore easily support these various streaming profiles.
4. Digital rights management (DRM)– For premium video like movies and sporting events, content owners want their rights protected. DASH is focused only on the core streaming technology and hence works seamlessly with various DRM solutions.
If you want to know more of the technical details, Thomas Stockhammer, on our team has published a paper on the design principles and standards we’re putting into DASH.
What’s Qualcomm doing?
Qualcomm and Qualcomm Innovation Center, Inc. (QuIC) have participated as the work-item lead and helped promote DASH with 3GPP, and were the main authors of the DASH specification in MPEG. In collaboration with companies such as Ericsson, Apple, Netflix and Microsoft, Qualcomm has worked on the standard. Although MPEG-DASH content has yet to be published, we’re working with content owners to help bring this open standard to market.
As a result, we’re building a lot of expertise and we’ve chosen to make it widely available. As a matter of fact, to encourage adoption of the DASH standard, Qualcomm will not seek royalties or license fees for use of its DASH Essential Patents as defined in our DASH Licensing Commitment.
You’re going to see similar announcements from other technology companies who realize that it’s time for an open standard for adaptive, Internet streaming video – one which is also easy to implement and bring to market.
Keep an eye on DASH as the standard evolves, and let me know in the comments what your company is doing about the user experience in mobile video.
Through its collaboration with Microsoft, Qualcomm is proud to show the Windows 8 Developer Preview running on the latest dual-core Snapdragon processor. Shown running on the desktop, Internet Explorer in the Windows 8 Developer Preview features support for the latest web standards as well as the Adobe Flash Player, giving consumers a rich browser experience and developers support for whatever tools that best suit their needs. Flash is an important part of the web browsing experience – and Qualcomm supports the Flash Player today on our dual core Snapdragon processor running Windows 8.
By optimizing the browser to really take advantage of dedicated hardware blocks in our Snapdragon mobile processors, we’re seeing comparable levels of video performance across both web apps and native apps – 30 frames per second. Not only can we play 1080p video files, we can playback 1080p video in Flash and HTML5. In fact, in HTML5 we’re able to get multiple video streams running live on a page at the same time.
We’re also closing the gap on advanced graphics with the help of HTML5 Canvas for 2D graphics and WebGL for the 3D equivalent. We’re seeing sample 3D WebGL content benchmarked at 25fps in a Web App, vs. 50fps in a native, OpenGL-ES equivalent app on the same device. Though the native app offers twice as many fps, anything over 25fps is not very noticeable to the human eye. Though we see this gap steadily closing over time.
I think we will see web apps first in tablets followed by Smart TVs. Once more and more TVs have full HTML5 browsers in them, it’s going to break open a whole new set of exciting user experiences. For example, you will no longer be tied to a limited set of Samsung TV Apps, LG TV Apps or Roku Apps. The whole Web will be at your beck and call via your TV Remote. That’s quite a game changer — one that Google TV has attempted to bring us.
If web apps become mainstream on tablets and TVs, they will have to become mainstream on allmobile devices. Speaking of which, we cannot discount the growth of the hybrid apps that are currently available on smartphones, which leverage a lot of HTML5. These are already mainstream.
Web apps are destined to take off for another simple reason: there are a lot more web developers (familiar with HTML5) out there than native app developers. And there are even fewer developers building tablet apps and TV apps. So the momentum is behind web apps — it’s just a matter of time.
The technology in our Snapdragon chips is always evolving, and we are constantly adding more intelligence and features into the chipset via hardware and software. The more features we add, the more we want to expose to web apps.
For example, we’re pushing the envelope in terms of what the camera can do with things like facial recognition, multi-shot with zero shutter lag, smile detection, blink detection, gaze analyzer, etc. So now, it’s no longer just about exposing a camera API to web apps, its about exposing all these advanced post processing features to web apps, too. Similarly we’re doing some very cool things around proximity-based peer to peer (P2P). Imagine the possibilities when your web app can reach out, discover and connect with other web apps nearby you.
Also, as I touched on earlier, we’re working to bring our Snapdragon processors to TVs, too. We suspect that more people will want to buy connected TVs that have all these cool HTML5 web app capabilities, as opposed to spending thousands of dollars and being locked into just Samsung, LG or Roku TV apps.
… with the Snapdragon chip, your browser doesn’t have to be just another piece of software. It can be optimized to take full advantage of all of Snapdragon’s subsystems. Here are a few examples of how a web browser’s performance can be turbocharged when tuned for the Snapdragon chip:
(Optimizations for the Snapdragon integrated modem and intelligent connectivity engine.)
- Designed to achieve up to 50% faster page and web app downloads1
(Leveraging smarter caching.)
- Improved multi-core utilization
(Leveraging Snapdragon’s GPU and multimedia hardware engines.)
- HTLM5 video performing at full native rate
- Faster and smoother scrolling, zooming and panning
- GPU accelerated HTML5 <canvas>, <video>, WebGL and CSS3D animations
1 Source: Tests performed by Qualcomm Innovation Center, Inc. Tested with 30 sites on Wi-Fi and consistent environment on Android 2.3 using HTC Sensation and production OEM device with Dual-CPU A9.
2 Source: Tests performed by Qualcomm Innovation Center, Inc. Tested using Android 2.1 through 2.3 on HTC Nexus One).