EuroPython 2009 Talks Abstracts
Talk Streams
- Python in Action
- Alternative Implementations
- Build, Packaging & Deployment
- Community
- Databases
- Documentation, Tools & Library
- Education
- Games Programming
- UI Programming
- Python Language
- Multimedia
- Network
- Science
- Testing
- Web Programming
Python in Action
ArcGIS and IronPython
How to use ArcObjects with Python to build GIS systems
Talk, 45 minutes (intermediate)
ArcGIS is a market leading Geographic Information System (GIS) software suite for Windows, Unix and Linux. It is built on ArcObjects, an all encompassing GIS library with bindings to .NET and Java.
In this talk I will briefly introduce ArcGIS and ArcObjects. I will then demonstrate how to use ArcObjects through IronPython. The talk will close with coverage of a real world application of these techniques.
Tags: GIS IronPython ArcObjects Geodatabase
Python ain't no snake oil
How can you persuade your peers and stakeholders to give Python a chance? We will look at: - Strategies of persuasion (that is: ways to persuade people to do things) - Obstacles of understanding - Learning from results of modern research of thinking - The business aspects of Python - Social aspects of Python - Why functional programming may be harmful - How licences may affect business
Harald Armin Massa (GHUM Harald Massa)
Talk, 45 minutes (intermediate)
There is far more to Python than just being free as in beer.
As well as the positive aspects on your mental health and your sexual powers, there are legions of effects that will not only make you more productive, but also make your stakeholders laugh with joy.
This talk will focus on the economic, social, systemic, business and sales aspects of Open Source and Python ; provide you with a wealth of arguments and strategies to promote Python with your customers, your peers and your organization.
Quite some time will be used to explore the ways people make decisions, on ways of persuasion; and on exploring stubbornness and learning to understand why people may be not be willing to welcome their superior Python overlords.
Open Object Framework
Rapid Application Development with Open Object
Fabien Pinckaers (Tiny sprl)
Talk, 60 minutes (intermediate)
Open Object (http://openobject.com) is an open source Python framework that lets you build your enterprise applications extremely fastly and easily. Open Object is the framework used by the leading open source enterprise management software: Open ERP (http://openerp.com).
It includes an ORM, a workflow engine, several report designers, a MDX engine, a dashboard designer, a module system, a process designer, an automated migration engine, web-services, and much more...
When you develop on Open Object, you get a web (Ajax) and a rich application (GTK & QT) with the same code. Open Object has also more than 350 modules dedicated to enterprise management: crm, sales, accounting, document management system, BI, etc.
In this talk, I will show how to develop a complete business application in a few minutes. The scenario will be based on a school management application need. The application will cover: planning of courses, management of students and teachers, different reports, workflow of courses, subscription and link to an internal documentation management system and a student portal.
Open ERP
Enterprise Management with Open ERP
Fabien Pinckaers (Tiny sprl)
Talk, 60 minutes (beginners)
Open ERP is a fully integrated enterprise management system. It contains 350 modules that covers: sales management, project management, marketing, hr, mrp, accounting, crm & srm, stock management, scm, etc. Open ERP also contains generic purpose modules like a wiki, a document management system, a BI, a webmail, etc. It is able to fit most enterprise needs with only one application.
In this presentation, I will present how Open ERP can help organising a complete enterprise and integrating all his services. The presentation will be in three parts: 1. An overview of Open ERP 2. An example on how to use Open ERP for two needs: a manufacturing industry and a service company. 3. How to customise your application using the screen designer, the workflow editor, and the report engines.
Python Coin: Making Money with Free Software
Python and the Netherlands' new 5 Euro commemorative coin
Talk, 30 minutes (general)
The Dutch Ministry of Finance organized an architecture competition for which a selected group of architectural offices (unstudio, nox, ...) and artists were invited, including myself. The goal of the competition was not to design a building, but the new 5 euro commemorative coin with the theme 'Netherlands and Architecture'. I am proud to state that I won the competition thanks to Python.
The whole design was done 100% with free software. The biggest part consists of custom software in Python, of course within the SPE editor. For the visual power I used PIL and pyCairo. From time to time also Gimp, Inkscape and Phatch helped quite a bit. All the developing and processing was done on GNU/Linux machines which were running Ubuntu/Debian. In the end I had to collaborate closely on location together with the technicians of the Royal Dutch Mint (coin factory). So all the last bits were done on my Asus Eee PC. (I am still wondering why Asus doesn't offer Ubuntu on its netbooks.) The Eee laptop took a bit longer (30 seconds instead of 3 seconds to generate a whole coin), but did the job just fine. For looking up the number of hits on the internet, I rediscovered Yahoo, which provides a much better api for automatic querying than its competitors. Of course the jury judged only the design and not the software used as others used Maya, Illustrator, ...
OSS, BSS and Python
Using Python as a telecom integration tool.
Tobias Rundström (Purple Scout)
Talk, 45 minutes (intermediate)
Python is a versatile tool, which is why we chose it to write integration software. This talk will outline how we (Purple Scout AB) managed to integrate big telecom systems like Cisco BACC, proprietary BSS and RADIUS servers to do authentication and billing. It will also highlight the components we are now giving back to the Python Community.
Jumping worlds with Python
A report on how we are using Python to implement a library for "native", Java and DotNET applications.
Thomas Bleier (Austrian Institute of Technology)
Bojan Bozic (Austrian Institute of Technology)
Talk, 30 minutes (intermediate)
When we were looking for efficient ways to implement our time series processing library that would allow us to use it from all major platforms (Java Runtime, DotNET runtime and native applications), we came across Python and it's implementations on these platforms (CPython, Jython and IronPython).
So we started the experiment to implement the library in Python, and tried to make it available for all those platforms.
The talk will cover - our approach - issues we discovered through this process - a few hints and guidelines for anybody trying to do something similar - current status and open issues
The visual_studio.vim Plugin
Python as a glue language in the visual_studio.vim plugin.
Talk, 30 minutes (intermediate)
This talk will describe the use of Python within the visual_studio.vim plugin for the Vim editor. It is an example of how Python can act as a bridge or glue between systems. The visual_studio.vim plugin is a way to interface the Vim editor to Microsoft Visual Studio development IDE. The vim plugin and Python module are hosted on the vim website at http://www.vim.org/scripts/script.php?script_id=864. The real work of the Vim plugin is achieved by a Python module which interacts with the Visual Studio COM interface.
Topics that will be covered:
- Description of the Python module that bridges Vim and Microsoft Visual Studio
- Python programming on Windows using the win32com module
- Using Python to interact with the COM interface of Microsoft Visual Studio
- Use of Python as an embedded DLL (as in Vim) versus Python as a standalone executable (the interpreter acting as main)
Test-O-Matic
A methodical approach to automated, full coverage software testing using Python.
John Cobo (British Airways)
Talk, 45 minutes (general)
Test-o-matic (TOM) is meant for integration/system/acceptance tests rather than unit testing.
- Test-O-Matic is a framework and a tool enabling the generation of software tests that are:
- robust - the tests should not break and if they do they are easy to fix
- automated - tests can run without human intervention
- methodical - a scientific approach to deciding what to test
- flexible - generate tests for soapUI, Selenium, and many more testing tools
- and provide full coverage - generate tests that get deep into the corners of your code base.
Test-o-matic (TOM) is meant for integration/system/acceptance tests rather than unit testing. It encourages the tester to work to edge cases and combinations of edge cases. In this way one avoids the trap of only testing what is easy to test and sure to pass.
Twittering with Python
Python libraries for the Twitter API: Overview and applications
Andreas Schreiber (German Aerospace Center (DLR))
Talk, 45 minutes (beginners)
The use of the Web 2.0 service Twitter is growing rapidly. More and more people, projects, companies, organizations, or others are using Twitter to send out a lot of 140 character messages to the world (i.e., their followers).
This talk gives an overview of how to use Python for automating Twitter actions, such as sending tweets, viewing tweets of friends, and managing the list of Twitter friends. The various existing Python libraries for accessing the Twitter API are presented and compared, listing advantages and drawbacks of each. This talk will teach you how to use the APIs.
The talk will also demonstrate a variety of use cases, including the use of Twitter in software engineering and software development, in distributed computing, on smart phones, or for system monitoring and present examples and hints for integrating Twitter into existing Python applications.
Mixing Python and Java
How Python and Java can communicate and work together
Andreas Schreiber (German Aerospace Center (DLR))
Talk, 30 minutes (intermediate)
Java is being used for many existing applications in a variety of domains. Sometimes it is useful to integrate them with Python. For instance, one may wish to add embedded Python scripting to Java applications, to communicate with separate Python code written for CPython, or use existing Java libraries from Python code.
The talk gives an overview of the various techniques, tools, and libraries for bridging Python and Java. In particular, the following implementations are described and compared: * Embedding of Python into Java with Jython. * Calling Java code from CPython scripts with JPype. * Calling CPython from Java code with JEPP. * Using inter process communication between Python and Java (e.g., with CORBA or SPIRO).
Examples for all techniques are demonstrated, and use cases for the different approaches are presented with examples from real world applications.
Python for System Admin
Using Python for multi-platform system administration type tasks
John Pinner (Clockwork Software Systems)
Talk, 60 minutes (intermediate)
The philosophy of Unix sysadmin was to have lots of small tools, each of which did one job well, and to provide the mechanisms to join them together to perform larger tasks. The glue joining them together was the Bourne shell. Generations of Unix system programmers used Bourne with the classic utilities like awk, sed, find and tar to look after their systems.
Over the years the development of Unix-like systems such as Linux has seen this philosophy diluted with the advent of the bloated bash shell and utilities, although based on the original Unix tools, with ever more complex options.
An alternative I have tried is using Python. Python is a compact programming language, augmented with a large number of modules which extend its capabilities. In this respect Python is similar in philosophy to classic Unix thinking, with the added benefit of being a modern high level and object-oriented language.
In this talk I will describe some of Python's capabilities in a sysadmin context, with code examples, and discuss its benefits. One of the greatest is that Python is multi-platform, being available on Unix, Windows, Linux and Mac. For example it is possible to have one backup script for use across all platforms, and to restore your backups across platforms.
Distributed Programming Made Easy with py.execnet
This talks discusses a very simple model for executing code in local or remote processes by using py.execnet. .. py.execnet: http://codespeak.net/py/dist/execnet.html
Holger Krekel (MerLinux GMBH)
Talk, 60 minutes (intermediate)
This talk discusses a very simple model for executing code in local or remote processes by using py.execnet. It allows one to instantiate a local or remote process for executing user-specified code fragments. Using channels one can continue to communicate with this code fragment. The instantiation of such remote processes does not require any installation on the remote side because all bootstrap code is sent to the other side. We first look into some examples of using it for sysadmin-tasks and then showcase new mechanisms for efficiently managing communication with multiple processes, contrasting this with the new "multiprocessing" 2.6 module and Remote-Method-Invocation approaches.
Python, .NET and the Weather
Integration of Python, IronPython, C# and ASP.NET with examples from an online weather derivative trading system
Talk, 30 minutes (general)
We will describe how the mechanics behind the weather derivative trading software of CelsiusPro.com works.
Weather derivatives are a special type of insurance against financial losses due to the bad influence of the weather. For example, the financial impact of a rainy weekend on an open air event could be reduced by a contract which pays the organizer a certain amount of money if it is raining on the morning of day in question.
The company CelsiusPro offers a trading system which allows companies and individuals to buy weather derivatives customized to their needs so that the financial impact of bad weather can be reduced. Part of the system is also available online and products can be bought automatically.
Obviously, such a business requires a sophisticated software environment. CelsiusPro with the help of soXes created a system using Python, Ironpython, C# and ASP.NET.
The authors believe that this system is a very interesting sample of an architecture which pragmatically combines different programming languages in a single application environment.
After some introductory samples of weather derivative products, the second part of the talk / publication will describe the architecture and show that thanks to the unique combination of pure Python, IronPython, C# and ASP.NET the CelsiusPro system is very flexible and easy to extend. Then a short demo will show how a pricing request can be done online.
To conclude, the experiences of the developers during the Implementation will be described and some practical hints about the limits of the used system architecture and the factors one must keep in mind when using Python / IronPython in combination with C# and the Microsoft .NET environment in general. For example, the development cycles differ and this can lead to big problems and inefficiencies.
RjDj + Python = Reality morph for the iPhone
How the RjDj team has used Python to augment their reality-enhancing iPhone app
Talk, 45 minutes (intermediate)
This talk will cover in depth how we have used Python and pushed Django to its limits in order to break our app out of its iPhone jail and into the network. We are using Python in all kinds of ways from web, to systems code, to desktop, with topics including:
How we saved time, memory, and cpu cycles by pushing our heavy content into Amazon's S3 cloud, and a once-over of the code (now open source) which allowed us to do this from inside Django.
The Python based tools we have built for our RjDj 'scene' composers, which allow them a simple, user-friendly way to get their Pure Data programs onto their devices for testing, among other things.
How we've created a single sign-on architecture by allowing users to log into our Django app using Facebook, Twitter, and other external authenticators.
The Python glue-code which we are using behind our crontabs to do near-realtime transcoding of Recordings uploaded by our users.
A discussion of future features in implementation and ideas we are considering, and how we will be using Python to implement these, including multi-player jamming with scenes, musical RSS feeds, game-like composition interfaces.
All of this with Python!
Working with Excel Files in Python
No need for COM, Python has two excellent libraries for dealing with Excel files.
Chris Withers (Simplistix)
Tutorial, Half day (general)
Too many people in the Python community think the only way to work with Excel files in Python is using COM on Windows.
This tutorial will introduce xlrd and xlwt, a pair of Python packages for reading and writing Excel files in pure Python that runs on any platform.
We'll go through the process of reading Excel files to extract data and formatting, writing Excel files and filtering existing Excel files.
You'll need a laptop with Python already installed to get the most out of this tutorial.
Feel free to bring along any Excel-related problems you'd like to solve as there'll be a section at the end just for this!
Kamaelia:Pragmatic Concurrency
Kamaelia makes concurrency a useful tool, rather than a pain - learn how!
Michael Sparks (BBC Research)
Tutorial, Half day (intermediate)
Why use concurrency? Since concurrency is viewed as an advanced topic by many developers, this question is often overlooked. However, many real world systems, including transportation, companies, electronics and Unix systems are highly concurrent and accessible by the majority of people. One motivation can be “many hands make light work” but in software this often appears to be false – in no small part due to the tools we use to create systems. Despite this, the need for concurrency often creeps into many systems.
Kamaelia is a toolset and mindset aimed at assisting in structuring your code such that you can focus on the problem you want to solve, but in a way that results in naturally reusable code that happens to be painlessly concurrent. It was designed originally to make maintenance of highly concurrent network systems simpler, but has general application in a wider variety of problem domains, including desktop applications, web backend systems (eg video transcode & SMS services), through to tools for teaching a child to read and write.
- This tutorial will cover:
- A fast overview in the style of a lightning talk.
- Kamaelia's core – Axon – which provides the basic tools needed for concurrent systems, followed by a session on implementing your own core.
- Practical walk throughs of real world Kamaelia systems to demonstrate how to build and design systems of your own.
- More advanced concepts such as reusing centralised services and constrained data sharing, as well as ongoing open issues will be touched upon.
- Tips, tricks and rules of thumb when working with concurrent systems.
During this highly practical tutorial, where you will create your own version of Axon, your own components and first Kamaelia based system (bring a laptop!). The course expects a level of familiarity with Python but no prior experience of concurrency is assumed.
We need to fail... and we need to fail quickly.
Learn an agile workflow used for Python/Javascript development.
Talk, 45 minutes (general)
One size does not fit all and this is true for agile methodologies as well. During the last two years I have been working with the developer team at Open End, facilitating the adoption of agile practices in their work flow (while in parallel continuing my involvement in PyPy).
During this talk I will address which agile practices we chose, why and how they are implemented. I will talk you through the full pipeline of how an agile work flow can look like and what the artefacts are.
Be prepared for cool product support for our work flow (Eutaxia), scenarios, gummy bears, tagging and the importance of failure.
This talk is for anyone who wants to get practical knowledge of an agile work flow, as validation for what you are doing right now, for inspiration of how things can be done or for learning what this agile stuff is anyway.
Psst: the talk title is a quote from my esteemed team colleague Samuele Pedroni. Credit and Kudos to him for being a great guy to work with.
Alternative Implementations
Introduction to IronPython
Introducing IronPython for Python developers.
IronPython is an implementation of Python, written in C#, that runs on the .NET framework and Mono. It is developed by Microsoft and was the first project to be licensed under their OSI approved Ms-Pl Open Source license.
Michael Foord (Resolver Systems)
Talk, 30 minutes (beginners)
In this talk we'll look at the differences between IronPython and CPython and why Python developers might be interested in using it.
We'll also see how to get started with IronPython and interact with types from the .NET framework.
HotPy - A High Performance Python VM
HotPy is a new high-performance virtual machine for Python 3. It runs'on the metal' like CPython and PyPy, and was designed from the ground up for high-performance. "HotPy" is a recursive acronym for "HotPy Optimising Translating Python".
HotPy's optimising interpreter is significantly faster than CPython and is g etting faster. HotPy can also translate byte-code to machine-code for further speed increases.
Talk, 30 minutes (general)
HotPy is a new virtual machine designed to run unmodified Python 3 code and supports all the features of a Python VM: full introspection, closures, generators, etc.
HotPy uses optimisation techniques based on type-profiling to achieve signficant speedups, and includes a just-in-time compiler which can provide extra performance for longer running applications.
At the moment it just a virtual machine, no batteries are included.
In this talk I will give an overview of the HotPy architecture, explain type-profiling and why it is important for languages such as Python, present some performance results and outline a plan to turn HotPy into a useful Python implementation.
Optimisation in HotPy
How the HotPy virtual machine achieves its performance
Talk, 30 minutes (advanced)
HotPy's superior performance is due the use of 'speculative' optimisations. Rather than modify code using knowledge that is provably true, as do normal compilers, HotPy improves code speculatively, using knowledge of what is probable. Some of this knowledge comes from a knowledge of the semantics of Python, but most of it is gathered by profiling at runtime.
This talk will discuss the theory of these optimisations, and give several examples of their implementation.
The merits of translation to machine code will also be discussed.
Ironclad: Hubris and Laziness
The lack of support for compiled CPython extensions is a serious problem for alternative implementations of Python. Ironclad is, so far as I know, the only major attempt to rectify this situation: it's a module for IronPython on Windows that enables binary compatibility with extensions compiled for CPython 2.5. These include large parts of the NumPy and SciPy packages, in addition to simpler modules like bz2, _hashlib, and _elementtree.
I plan to discuss what I have done to get this far, what strategies succeeded and failed, and what I've learned that may be valuable in implementing similar functionality for PyPy and/or Jython.
Talk, 30 minutes (advanced)
This talk is primarily concerned with the details of hooking up parts of two separate Python implementations -- CPython and IronPython -- such that objects from either one can be freely used by the other without anyone needing to recompile anything. Most of the discussion will be at a relatively high level, but pitched towards those with some interest in the details of different Python implementations.
Among other topics, I intend to discuss the various strategies I have used to represent different kinds of objects in foreign runtimes; the impact of CPython's Global Interpreter Lock; how to bolt a non-deterministic garbage collector on top of CPython's nice simple deterministic one; and the interesting problems surrounding files. Insults to static type systems will be kept to a bare minimum, but attendees should be prepared to witness explicit descriptions of ugly hacks performed by consenting adults.
Despite the fact that a great deal of Ironclad's code is already autogenerated, an awful lot more of it could and should be. I will finish by discussing the code generation as it currently stands and how I plan to expand it, and specifically address Ironclad's potential as a base for similar efforts targeting PyPy and/or Jython.
A Better Python for the JVM
How we are making Jython a high performance implementation of Python.
Talk, 45 minutes (advanced)
For future versions of Jython we are working on an optimizing compiler that applies many traditional compiler features to the dynamic nature of Python. The aim of the project is to replace the current compiler in a version after Jython 2.5.
In this presentation I will talk about the optimizations performed by this compiler, both static optimizations as well as optimistic optimizations with fallback capabilities. I will also share a comparative overview of the performance of Jython with the new compiler compared to the old compiler and CPython.
This will be a fairly advanced talk about the prototype for an optimizing compiler for Jython. I will present the general structure of this compiler, the optimizations it performs and the performance improvements it gives.
PyPy - Complete and Fast
An update on the state of PyPy, the Python implementation in Python.
Samuele Pedroni (OpenEnd AB)
Antonio Cuni (PyPy / University of Genova)
Talk, 60 minutes (intermediate)
Part 1 - becoming complete - 30 minutes of the projected 1 hour
PyPy has come a long way. No longer merely a research project and proof of concept, PyPy now is able to run sophisticated python applications such as Django and Twisted. We will focus on the improvements that happened during the last year in PyPy, which made this possible.
Thus -- even before the JIT work -- we could assert that PyPy now provides a complete, flexible and reasonably fast Python interpreter, itself implemented in Python. ("Complete" means it is fully compliant but misses a lot of third-party extension modules.)
This section of the talk is mostly intended for people who are already familiar with the PyPy project, and want an update as to how we are doing. We will also explain the motivations driving current PyPy development and its targets for the future.
Part 2 - becoming fast - 30 minutes of the projected 1 hour
As we write this abstract we are finally integrating our Just-in-Time compiler generator into PyPy. Preliminary results look very good (more during the talk!).
The PyPy JIT compiler, in addition to being a specialising compiler, is now a tracing compiler. Another example of a tracing compiler is JavaScript's TraceMonkey, soon to be included in FireFox. We will give the status of the implementation of our JIT, with speed measures.
We will then describe how it all works. No familiarity with tracing or specialising compilers required.
Due to our architecture we can generate code for C but also for the JVM and .NET; it is possible to generate a JIT for these platforms too, running on top of the platform's native JIT.
Also, as usual in PyPy, one can generate such a tracing compiler for any computer language, only by writing an interpreter for it. We already have ones for Squeak, Prolog and GameBoy, and experimental ones for JavaScript, Scheme, etc. so people wishing to speed up other languages now have a clear and simple path to follow.
Talk by: Antonio Cuni, Carl Friedrich Bolz, Samuele Pedroni, Armin Rigo
Psyco V2 - Generators Welcome!
Psyco now has generators and a bunch of other features
Talk, 45 minutes (general)
Psyco is a specializing compiler for Python, that was written by Armin Rigo beginning in the year 2000 and was under active development until September 2005. Since then, Armin moved on to developing PyPy and only made occasional maintenance releases of Psyco.
This was disappointing for Psyco users, who would have liked it if psyco continued to evolve as Python changed. They were very interested in psyco because it provided them with needed, necessary speedups essential to their business.
Starting with creating and funding the Need For Speed sprint in 2006, John Benediktsson and David Salomon. from EWT, now FATTOC LLC decided to continue the development of psyco with the intention to give the results back to the community.
Psyco V2 is the result of these efforts. It has been released as open source XXX what license, Christian? and is under active maintenence and support by me (Christian Tismer).
Some of the new features include a couple of newly accelerated builtins, the ability to run under Python 2.4, 2.5, and 2.6, replacing builtins by external psyco accelerated Python code, compact objects are now safe and give incredible extra speed, and finally generators are now fully supported!
This last point will be a major focus of the talk, because getting generators to work was much harder than expected.
Besides new features, we will learn about the difficulties during development, especially debugging, and tiny errors which were hiding for years.
We will also talk about missing features which are likely to be added, such as nested scopes, or quite unlikely, such 64 bit support.
Python VM Panel
Question and answer session on the multiple Python implementations.
Michael Foord (Resolver Systems)
Talk, 45 minutes (general)
There are now many different implementations of Python, some complete, some still in progress and some which are really different languages that happen to look a lot like Python.
If you've ever wondered why you should even consider using IronPython, or if there are any benefits to switching to Jython - or even secretly suspected that all these different versions just fragment the community and confuse users - then this is your chance to harass and harangue those responsible!
Linker: The “bolt-on JIT”
Somewhere between CPython and the JVM
Talk, 30 minutes (intermediate)
CrossTwine Linker (http://crosstwine.com/linker/python.html) is not a virtual machine. Nor is it an execution engine. It is a growing toolkit of C++ components implementing bits and pieces of modern virtual machine technology, which can be combined using policy-based design to augment an existing interpreter, yielding substantial performance improvements while retaining compatibility (including with “C” extensions) and following language evolutions.
This talk will present some of the ideas, technical details, and methodologies underlying the development of our derivative of the CPython interpreter, explain some of the challenges of the current mainstream implementation and how we work around them (or don't), and discuss how this project relates to other efforts to improve Python.
Build, Packaging & Deployment
SCons the Builder
Dynamic programming languages are changing the way in which build is undertaken
Russel Winder (Concertant LLP)
Talk, 60 minutes (intermediate)
Make was a revolution and a revelation. It changed the world of compilation and document preparation. Autotools and CMake try to keep Make the supreme build tool but the cracks are appearing. Ant and Maven took a different tack and, in the Java world, they took over as the only tools of build.
The dynamic programming languages change the balance: from external DSL (domain specific language) to internal DSL. SCons is a tool that shows how the dynamic nature of Python changes the whole way in which build is managed. Build is process, SCons allows it to be programmed.
Compiling Python with Byte Coat
How to protect your Python modules from decompilation by translating them into C Extension Modules
Python uses the concept of packages and modules to organize the source code of applications, the standard library and third-party libraries. The modules can be distributed as plain source code, precompiled byte code or as C extension modules which must be compiled natively for the target system. Distributing only byte code seems like a safe way to prevent users from seeing proprietary source code. However, it is possible to extract nearly all of the source code from the byte code. There exist commercial tools to achieve that, such as decompyle.
Some approaches like Pyrex, Cython, etc. try to primarily increase Python execution speed by translating Python code to similar C code. This also solves the mentioned security problem but is limited to a restricted subset of Python.
We developed a method to automatically convert byte code modules to corresponding C extension modules that can transparently replace the original ones. This is achieved by translating Python byte code into equivalent C code, that contains the same C code the interpreter executes when encountering a corresponding byte code instruction. This involves using the C Python API so that we gain almost full compliancy and security.
Talk, 45 minutes (intermediate)
First, I will talk about our motivation for writing Python Byte Coat and what other approaches we examined to avoid distributing Python source code or revertible byte code.
After this, I will shortly explain why we developed a new tool and why we did not use existing ones.
In the next section, I will talk about the current focus of the project and our goal for future development.
In the following section, I will explain the technical aspects of the tool and some of the problems we had to solve.
After this, I show typical usage scenarios and real world examples of Python Byte Coat.
I will close the talk by enumerating advantages and disadvantages of our approach and in which way further development could go.
Community
Corporate FOSS
Why and how to interact with FOSS communities?
Tobias Rundström (Purple Scout)
Talk, 30 minutes (beginners)
This talk is about FOSS at your work place. Do you want your company to be more FOSS compatible? Do you want to promote your company with FOSS products. Or maybe you just use FOSS at your company and you want to give something back? This is an orientation, what do you need to know and a good encouragement to do it!
Things I Helped Create
What did I help create, why, and how?
Talk, 60 minutes (intermediate)
This is a wide-ranging philosophical talk that considers the processes of creativity and open source community. The talk is intended to be inspirational in nature.
The talk has two goals:
- Discuss a range of public Python-related things that I helped create. This includes software, projects, and other entities. Some of them might be interesting to you.
- Discuss the process of creation itself using these as examples, in order to extract some lessons from the successes and failures.
The Coder's Dojo
how to become a better software developer
Tutorial, Half day (general)
“I'm not a great programmer, I'm just a good programmer with great habits” - Kent Beck.
From the guy who wrote “Smalltalk best practice patterns”, invented eXtreme Programming, Test Driven Development, and JUnit, you might think this is just false modesty. But I disagree. Good habits of writing clean, easy to real code, with good automated test coverage, will get you a long way in this business. So what are you doing to cultivate your coding habits?
One thing I have found very helpful is to practice with “code kata” exercises. This is an idea introduced by Dave Thomas in his blog. He says “A kata is an exercise in karate where you repeat a form many, many times, making little improvements in each. The intent behind code kata is similar.” So you sit down for an hour or two and solve a small toy coding problem, a code kata. The challenge is to follow a good coding process – use small steps, work test driven. The code should build up from nothing into an elegant, well tested design. The next time you tackle the same kata you should start over, perhaps use different tools, implement the tests in a different order, try a different design. The idea is to practice on code that doesn't matter, and to reflect as you do it.
This kind of individual practice can help you to improve your skills, but what I have found even better than that, is to do it together with others. This is an idea developed by Laurent Bossavit and Emmanuel Gaillot in their conference workshop at XP2005. If you want to learn Karate, you go along to your local dojo and practice together with like minded people. So that's what this session is about. Code Kata in a group.
Suggested outline: 20 minutes – introductory presentation, background 60 minutes – prepared kata demonstration of the technique (KataMinesweeper) 60 minutes – randori kata, everyone involved (KataReversi) 20 minutes – debrief, reflection.
Snakebite: The Open Network
An action-packed overview of the network destined to shape the future of open source development.
Why do we develop open source software on closed networks? Why do open source developers only have access to a fraction of platforms that their software will be eventually be run on? And why are the Windows buildbots always red?!
What initially started as a devious plan to try and con either Neil Norwitz or Guido van Rossum into hiding some aging AlphaServers under their desks at Google, Snakebite now has its sites firmly set on rewriting the rulebook for how we develop open source software.
Snakebite is a collaborative development network intended to make the lives of Python developers easier and make cross-platform development less problematic. As well as changing the way Python is developed, it will simplify development for other Python projects like Django, NumPy, SciPy, Twisted, Trac, etc.
Talk, 45 minutes (general)
Join Trent “perhaps-I-went-slightly-overboard” Nelson, Snakebite’s founder, for an action-packed adventure through the self-proclaimed “Open Network’s” tumultuous past in the section I Wish I Was a Basket Weaver in Tibet: The Snakebite Early Days.
Gain an understanding of where Snakebite is today as Trent employs an arsenal of judicial exaggeration and strategic white lies to leave you feeling warm and fuzzy about the network in his overview, State of the Wound. Then, sit in awe as Trent demonstrates his inability to learn from the pitfalls associated with being too ambitious in the early days as he depicts his even more ambitious plans for the future in uninspiringly-titled section, The Road Ahead.
GPG Keysigning
Get set up with GPG Keysigning here
Meeting, 30 minutes (general)
An informal session explaining how GPG works and how to set it up - swap keys with other delegates and build your Ring of Trust.
Unconference Panel
Panel discussion on holding 'unconference' or 'barcamp' style events
Steve Holden (Holden Web, LLC)
Panel, 45 minutes (general)
This session concentrates on how a different style of event is held - the so-called 'unconference', 'bar camp' or 'open space' event where the programme is 'organised' by the delegates themselves on the day. We hope that both experienced and novice barcampers will join in to see how these events work and learn their strengths and weaknesses compared with a conventional conference like EuroPython.
Panel members will include Michael Sparks, Bruce Eckel and Christian Scholz and the session will be chaired by Steve Holden.
Databases
Introducing FluidDB
The database with the heart of a wiki
Terry Jones (Fluidinfo)
Talk, 60 minutes (intermediate)
FluidDB is a hosted database we (Fluidinfo http://fluidinfo.com ) will launch - by hook or by crook - into Alpha shortly before EuroPython 2009.
Currently there are only a few hosted "cloud" databases available, with Amazon SimpleDB and Metaweb Freebase being the best known. Other companies, like Microsoft, are scrambling to catch up. For the most part these databases are quite traditional and uninteresting, essentially offering SQL in the clouds.
Heart of a wiki
FluidDB is quite different. We say it's "the database with the heart of a wiki" because it's writable by any application. That's to say that when an application (or user) runs into an object in FluidDB, they can always add information to it. No need to ask permission, no need for anyone to anticipate needs or to decide in advance how information should be structured, organized, or combined. There are no database schema, and there is no distinction between metadata and data. Unlike a wiki though, there are permissions on object attributes to stop you from damaging existing content.
Why is this interesting?
Having a database that's fully writable in this way changes several important things:
- It gives programmers the flexibility to easily change applications in unpredictable ways after they have been deployed.
- It means that a successful application (e.g., Twitter) does not need to write a specialized restrictive API for 3rd party apps - they use the FluidDB API just like the original app.
- It means 3rd party apps can store their data together with the data of the original app (imagine being able to add ratings to Twitter tweets, and then search based on rating and tweet content).
- It allows us to build a family of apps that can share data. In this way it makes it simple to build mashups that add information to the world and then put it somewhere useful (i.e., onto the original objects), enabling more mashups instead of making further hoops for subsequent programmers.
But wait, there's more!
Allowing anyone or any application to add information to objects also has a big impact on control and organization of information.
- Control: UIs and APIs provide people and programs with access to information. But OTOH they also limit what we can do. FluidDB changes this because it allows search on anything - including whatever attributes a normal user or app has put into the system. So you might do a search for Amazon books that are out in paperback, and which have been read by Fred, and which have been mentioned in Slashdot, and which Sally has looked at on Amazon but not bought. That kind of search is impossible today because we do not have an underlying information architecture that lets us put all that disparate information in the one place and search on it. In FluidDB no-one can stop anyone from adding whatever they like to objects, or searching as they please, so this sort of thing is easy.
- Organization: Programmers have traditionally used data structures and pointers to organize information. While fast at run-time, that approach is very rigid. People, especially different people, want to organize things differently, or in multiple ways, and on-the-fly. Because FluidDB allows you to tag anything you want, you can use unique tags and search instead of data structures fields and pointer following. E.g., want to know what's in a folder? Search for objects tagged as belonging to the folder. You can build all data structures in this way, and these organizations can co-exist simultaneously without interfering with one another.
Outline
The timings shown here are approximate. I've left 5 mins free for questions.
- Intro - what is FluidDB, what's wiki-like about it? (10 mins)
- The (HTTP/REST/JSON) API. (10 mins)
- Example Python code and applications. (8 mins)
- The query language. (5 mins)
- How FluidDB affects:
- Building mashups - why it just got so much easier. (4 mins)
- Information representation - dynamic data structures and organization. (4 mins)
- Programmer freedom, control, and API/UI design. (3 mins)
- Programming - apps can let users own their own data. (5 mins)
- Applications that can work together, augment, and bootstrap each other. (2 mins)
- How to get Alpha access to FluidDB. (4 mins)
Parting thoughts
Wikis changed how people can create content online. FluidDB, I hope, will do a similar thing for applications and the people who use them. It has the wiki advantage of writability, but does not have the problems of wikis.
As you might have guessed/hoped, FluidDB is written entirely in Python.
We've open-sourced some core components already, and we will open-source the whole thing as soon as that's feasible. It's currently running on top of Amazon EC2 and S3 - an interesting story in itself.
I do not plan to talk about our company at all. I.e., this is not a commercially-oriented talk. We've been hacking on this for years as a tiny 2-person startup, with very little money. We think it's cool, we're programmers, we love Python, FluidDB is something for programmers, and the talk is for a programmer audience!
Search with Python and Xapian
Integrating a customised high performance search.
Richard Boulton (Lemur Consulting)
Talk, 45 minutes (intermediate)
Xapian is a mature and powerful search engine library, with comprehensive Python bindings. However, it is quite low-level and it's not always easy to get the full potential from it.
I'll describe a set of components built on top of Xapian in Python which make it easy to integrate search with any application. Amongst others, I'll describe the Xappy Python wrapper for direct integration, and the Flax Search Server which provides a web service for indexing and search. I'll give examples of websites using these components, including one using a neat Django integration layer.
Persistent Objects with ZODB
Don't jail your objects in an RDBMS, see how to use a native object database
Christian Theune (Gocept)
Tutorial, Half day (intermediate)
The ZODB is a mature object database for Python which requires very little effort from the programmer to persist its model objects.
We will look at how to develop an application using ZODB as well as how to maintain a ZODB database.
MongoDB in the "Real World"
A review of MongoDB and other similar datastores. Real world examples using MongoDB.
Mike Dirolf (10gen)
Talk, 45 minutes (intermediate)
I'd like to start by briefly discussing the paradigm shift that has recently been occurring in the database world. This will touch on CouchDB, MongoDB, Big Table, SimpleDB and other non-relational databases that have sprung up in the past couple of years. Will mention the use cases that have caused these databases to be developed and briefly discuss the differences between them.
Next I'd like to get into discussing real world use cases for non-relational databases. Talk about where they really shine relative to SQL. I'll be using MongoDB as a concrete example but many of the things I talk about will apply to other non-relational datastores as well, and I'll be sure to mention those cases.
This general discussion will be followed by specific examples. Here I'll show live demos using MongoDB and the code that is powering them. These examples will highlight use cases discussed previously. I'd like to spend a good amount of time going through the code and talking about the design and implementation of systems using non-relational databases (specifically MongoDB). This will be pretty technical - hence the Intermediate target audience.
Finally, we'll wrap up with a summary of what has been discussed and some Q&A.
Documentation, Tools & Library
Pythonic Filesystem APIs
Status updates.
Talk, 30 minutes (intermediate)
All these filesystems are yours, except Europa. Attempt no mounting of it. Use them together. Use them in Python.
A continuation to "My God, it's Full of Files" from EP2008, including status update and plenty of examples using the API.
repo = gitfs.repo.Repository(path)
with repo.transaction() as root:
with root.child('greet').open('w') as f:
f.write('hello, world\n')
What happened last year:
- http://eagain.net/blog/2008/07/07/ep2008-pythonic-fs.html
- http://eagain.net/blog/2008/07/19/europython2008.html
- http://eagain.net/blog/2008/08/11/ep-talk-videos.html
- http://www.europython2008.eu/FilesystemSprint
PLY and PyParsing
A comparison of the two most popular Python parsing systems
Talk, 30 minutes (intermediate)
Got a text parsing problem? In most cases using string manipulation and regular expressions will solve it for you. But when the input is complex, try reaching for a parsing system to help out.
The two most popular in Python are PLY and PyParsing. PLY follows the lex/yacc tradition with a domain specific language to describe the tokens and grammar. It was built with both error diagnostics and performance in mind. PyParsing is a recursive descent parser which expresses the format as a Python data structure. It make no distinction between lexer and grammar and has a uniform callback system which makes certain types of data extraction very easy.
In my talk I'll show the basics of how to use both systems for several different format parsing tasks, of different complexity. This will let you see how to use the parsers and understand more of the tradeoffs between complexity, readability, error handling, and performance.
Improving Client-side HTTP
Improving the available HTTP client libraries: discussion and future
Talk, 60 minutes (general)
The current functionality of the Python standard library in supporting the client side of HTTP communication is rather rudimentary (httplib, http.client in Python3 and urllib2). Furthermore, we are not aware that any external libraries exist that support our use cases. Via an introductory talk and further discussion, we hope to be able to define a roadmap for the future of HTTP clients written in Python. First, we will define our requirements, then we will explain the shortcomings of both the internal and external solutions, and finally, we hope to create a lively discussion about the future of HTTP clients.
FilterPype framework
FilterPype is a pipes & filters framework, using coroutines, simulating synchronous multi-threading.
Rob Collins (Folding Software)
Talk, 45 minutes (intermediate)
This talk will start with an introduction to the "pipes and filters" design pattern, to coroutines, and why writing small units of decoupled functionality is a good idea. We will see how easy it is to write coroutines as data filters, and link them together into a powerful pipeline. We will see demonstration applications, and discuss the best way to encourage use and further development.
FilterPype is being used for multi-level data analysis, but could be applied to many other areas where it is difficult to split up a system into small independent parts. Some of its features:
- Advanced algorithms broken down into simple data filter coroutines
- Pipelines constructed from filters in the new FilterPype mini-language
- Domain experts assemble pipelines with no Python knowledge required
- Sub-pipelines and filters linked by automatic pipeline construction
- All standard operations available: branching, joining and looping
- Recursive coroutine pipes allowing calculation of e.g. factorials
- Using it is like writing a synchronous multi-threaded program
Education
turtle.py - a Teaching Tool
How to Use Python's New Turtle Module for Teaching Programming.
The audience I want to address is not only teachers and educators, but also pythonistas who know teachers and other people in the educational area they want to convince to use Python as a teaching language.
Talk, 45 minutes (general)
Since the release of Python 2.6/3.0 Python has had a new turtle module as part of its standard library. It is an excellent tool for teaching and learning programming, not only for young children and high school students, but also at the undergraduate university level.
A few months after the release of Python 2.6 the module is not widely known yet. My intention is, to show how and to what ends it can be used in an educational setting.
In fact, the module not only offers turtle graphics in the traditional sense but a wealth of different ways ways to use it, thus not constricting the teacher's design decisions for their programming course.
Other important goals when developing the module were ease of use and implementing smooth yet user controllable animation, thus providing clear visual feedback to the students about the correctness/flaws of their programming ideas.
The talk will mainly consist of a live demonstration, in which I'll show ...
- ... how to get started with the turtle module - doing interactive graphics
- ... how to configure the module for classroom use according to the teachers preferences
- ... how to use it in a procedural programming style approach ...
- ... as well as in an OOP style approach ("objects first")
- ... how to create event driven programs
To show the broad range of possible applications, I'll finally walk through a series of examples, including some games, from very easy ones to ones using fairly advanced language features.
The MOAI Server
MOAI is a server platform that provides academic services for institutional libraries. It uses the Protocol for Metadata Harvesting defined by the Open Archive Initiative (OAI-PMH) to aggregate and exchange publication metadata.
Kit Blake (Infrae)
Talk, 30 minutes (general)
The software can harvest institutional repositories and (re-)publish the metadata and assets in multiple feeds in a flexible and easily configurable manner. Besides OAI-PMH there are developments to add SRU/CQL support and provide support for OAI-ORE content maps
The presentation will describe the MAOI server and its capabilities, how it's installed, configured, and what services it provides. The output of the MOAI server will be shown using components developed for the Silva CMS. The demonstration will show embedding of feeds in documents, feed configuration, and search facilities.
Software Apprenticeship
Teaching Software Development in Apprenticeship-style using Python
Christian Theune (Gocept)
Talk, 30 minutes (general)
For a couple of months I've been teaching software development to three apprentices using Python as the primary development language.
The idea of teaching software in a master/apprentice-style has been popping up in various articles before and I want to share experiences with an actual incarnation of this.
Using Python's Turtle to Teach
With version 2.6 Python gained a new, greatly enhanced turtle module. This module was primarily designed as an educational tool but it also provides quick and easy access to graphics – even interactively. The new module, in fact, supports much more than merely classical turtle graphics. It also supports different approaches to geometry, e.g. Cartesian geometry and working with geometrical objects as well as creating simulations and event driven programs, particularly games.
In this tutorial, we'll cover:
- How to use the turtle module interactively and how to configure it according to ones preferences and needs.
- A couple of ways to use it for introductory programming courses. As the module can be used equally easily with different programming styles, you will get some feeling of how to use it in accordance with your educational needs and/or your preferred programming style.
- Most of the new features of the turtle module. Each of the new features will be introduced by some small interactive "micro-experiment" revealing it's essence and it's working.
- Some a bit more elaborate exercises will be provided, designed to create small but useful scripts.
- Additionally a few demo scripts will be presented, showing a broad range of possible applications from simple to quite advanced.
Attendees should bring a laptop equipped with Python 2.6.
Tutorial, Half day (general)
Outline
Each of the following sections (except the first one) will be introduced with a short interactive "micro-experiment". These are not mentioned individually. Exercises are meant to be done by the attendees. The examples mentioned below (in some sections) will be demonstrated and explained. They may have different backgrounds – most of them are taken from my educational work with high school students as well as with teachers. Some are the result of trying to combine special language features of Python with graphics, and some are simply products of my exploratory curiosity.
Part 1 - Basics
- Introduction.
- Origin and characterization of turtle graphics
- The history of Python's turtle module
- Getting started.
Doing interactive turtle graphics
Configuring IDLE (why use the -n switch)
- Basic turtle graphics functions –> new features also here:
- improved animation,
- appearance of the turtle, various built in turtle shapes,
- shortcut aliases for most basic functions (forward –> fd, etc.)
- pencolor(), fillcolor(), color()
- hideturtle(), showturtle()
- undo() especially useful for doing interactive graphics with kids.
What about compatibility with the old turtle module?
- Configuring the turtle module
- The content of turtle.cfg
- Configuring the graphics window
- Configuring the turtle appearance
- What is the difference between standard-mode and logo-mode?
- Using translated doc-strings as help-texts: How to prepare an appropriate docstring file. (Optional – if there are attendees with foreign native languages)
- Turtle objects. (Object based programming).
- Using multiple turtles.
- Exercise: a turtle chase.
- The Screen object
It is returned by the Screen()-function: A singleton.
- Methods concerning the appearance of the graphics window
- bgcolor(),
- bgpic(), and more ...
- Methods affecting the behaviour of all turtles
- colormode()
- clear(), delay(), tracer(), update() ...
The relation between turtle functions and Turtle- respectively Screen- methods. (2 min.)
Part 2 - More advanced features of the turtle module
- Doing recursion.
- Trees
- Exercise: The "classical" depth-first tree. Visualize recursion-depth by the turtle's color.
- A breadth-first tree using the clone() method for creating a turtle for each branch – ending up in 2^n turtles (n being the recursion depth).
- An animation: the towers of Hanoi
- This example makes use of transformations of the turtle-shape (in this case: squares) through the method shapesize(). Thus the discs of the towers of Hanoi can be represented by turtles – and the towers themselves by lists of turtles.
- Doing Cartesian graphics
- goto(), setheading() and related methods.
- Setting world coordinates
- Example: plotting graphs of functions
- Exercise: plotting iterations of quadratic functions (Verhulst dynamics). This example will be very instructive concerning computer arithmetic.
- Make your own turtle shapes: (10 min)
- create, register, use
- simple polygonal shapes
- compound shapes consisting of several polygons (different colors)
- gif images as turtle shapes
- Create patterns by stamping turtle shapes.
- Using the Vec2D class (10 min)
- Example-Demo: creating simulations of physical systems – planetary orbits, a three/many body system.
Part 3 - Creating/extending turtle graphics applications:
- Event driven programming - Creating a drawing program.
- using mouse-events - onclick(), onrelease(), ondrag()
- using keyboard events – onkey(), listen()
- Putting things together to create simple games.
- Exercise: Greg Ewing's tile flipping puzzle
- A simple arcade like game:
- This is a spaceship-game demonstrating (to the player) the working of Newton's second law of motion in space. (Sort of 'lunar lander' variant)
Flickr for formulas
How I used Django and Pinax to build a social web site for sharing formulas.
Talk, 30 minutes (general)
MathTran is both a website and a web server that provides TeX rendering of formulas. Flickr and others are social networking sites for photos.
This talk describes how we used Pinax (a social web platform built on Django) and the MathTran web service to create a social networking site for mathematical formulas.
Games Programming
QCake 3d Game Maker and Python
News and short demo of how to use the QCake 3d game maker with Python
Talk, 30 minutes (intermediate)
QCake ( http://www.qcake.org) is an open-source, qt-based 3d game maker and includes scripting 3d scenes with Python.
I will show QCake, demonstrate how to use it, present news from the QCake developer ( Harald Krippl ) and demonstrate how to script a simple 3d scene using the Python scripting module for QCake.
I hope to get some people interested to help with developing QCake.
UI Programming
Pyjamas: Simple RIA Web apps
How to use Pyjamas to drastically simplify AJAX RIA development
Tutorial, Full Day (intermediate)
When people hear the word "AJAX" they cringe, and think of a ready-made highly static framework with limited specific functionality which they can never expand or hope to understand, which will never fully do the tasks they need.
Pyjamas allows a refreshing and radically different approach to AJAX programming: it allows the programmer to think, design and code web applications entirely in Python, yet the applications will still actually run natively in every single major web browser (no plugins required: only javascript).
Pyjamas is a python-to-javascript compiler and widget set that effectively turns the web browser into a desktop widget platform, thanks to the DOM model.
By manipulating the DOM model direct, Pyjamas startlingly puts web browser technology on a par with Python-GTK2 and Python-Qt4, with the added advantage over-and-above these "native" desktop widget sets of having complete and full access to SVG 2D Canvas, native javascript, HTML, CSS and NPAPI plugins such as Flash.
The tutorial will therefore go through the basics of creating web applications that look like desktop applications. Connecting to a Web Server will be covered, first using AJAX to load straight text, and then using JSONRPC to talk to Web frameworks such as Web2Py and Django.
Introduction to PyObjC
Creating Mac OS X native applications with Python and PyObjC
Talk, 45 minutes (intermediate)
Learn how to create native Mac OS X applications using PyObjC.
Learn all about the Python <-> Objective-C bridge and how you can use it to create first-class GUI applications for Mac OS X. No prior Objective-C or Mac GUI knowledge required.
Reusable components in PyGTK
Tutorial for intermediate level PyGTK development focusing on reusability and preventing spaghetti.
Tutorial, Half day (intermediate)
Often, GUI applications start simply and develop complexity. Extra features are added on, and the asynchronous nature of UI programming ensures that the application becomes difficult to understand.
This tutorial will teach people who have some experience of PyGTK how to develop applications using reusable subcomponents.
The tutorial schedule will be:
- 30 minutes discussing the nature of the problem
- 60 minutes writing some reusable components.
- 60 minutes experimentation in assembling the components into applications.
Requirements:
- Basic Python/PyGTK knowledge
- Laptop computer
- Installed Python, PyGTK
Python Language
Developing with IronPython
A tutorial on "Developing with IronPython": Python for .NET and Mono.
IronPython is a faithful reimplementation of Python, written in C#, that runs on the .NET framework's Common Language Runtime (CLR) and on Mono. Although the language is faithfully reproduced, the environment of the CLR is very different to CPython, with a powerful Just-in-time compiler, different garbage collection and an arguably superior threading model. Most importantly, .NET's huge range of libraries can now be used from Python.
IronPython is developed by Microsoft and was the first project to be licensed under their OSI approved Ms-Pl Open Source license.
This tutorial is an opportunity for Python programmers to get hands-on experience of application development with IronPython. Several major .NET API's will be covered and attendees will put this into practice by extending an example application.
Topics include:
- Differences between IronPython and CPython, including "Why Use IronPython?"
- Introduction to the .NET framework - a dynamic language on a statically typed frameworks
- GUIs with Windows Forms
- Databases
- Network requests and web services
- Handling XML
- Threading
Come with laptop - running Linux, Windows or Mac OS X - and ready to code!
Michael Foord (Resolver Systems)
Jonathan Hartley (Resolver Systems)
Tutorial, Half day (intermediate)
Detailed summary:
"Why use IronPython?"
- Differences between CPython and IronPython
- Advantages of IronPython
- Disadvantages of IronPython
Introduction to .NET
- Working with the Common Language Runtime (the clr module)
- .NET libraries - namespaces and assemblies
- .NET datatypes (the basic Python types and .NET specific types like enumerations and delegates)
- A dynamic language on a statically typed framework (arrays and generics, method overloading)
- The Python standard library from IronPython
- The IronPython interactive interpreter
GUI Development with Windows Forms
- Subclassing 'Form'
- Standard 'Controls' (panels, buttons etc)
- Introducing 'Stutter' - the example application
- Layout and event handlers
- Menus
Making network requests
- The Twitter API (a RESTful API)
- The WebRequest class
- .NET streams
- POST and GET
Parsing XML
- The XmlReader class
- Basic parsing
- A cleaner 'SAX-like' interface
- Parsing Twitter XML
Databases
- ADO.NET
- Data provider layer
- Connections and commands
- Avoiding SQL injection
- Reading the results
Threading
- A different threading model
- Creating threads
- Aborting threads
- The CallTarget0 delegate
- Windows Forms and threads (syncing back to the control thread)
What Sucks About Python
Python stole all its best bits from Lisp/JavaScript/Fortran/MFTL and you should go learn those.
David Jones (Ravenbrook Limited)
Talk, 60 minutes (intermediate)
A lively discussion on what bits Python stole from other languages, and what other languages should steal from Python.
Python is a dynamic language with support for object orientation and functional programming. Just like Perl, Lisp, Lua, JavaScript, and so on. And these languages are basically the same, right? Lists, Hash-tables, first-class functions, objects, late-binding. The differences are a just a bit of syntactic sugar. Clearly Python is the best, but can it made better? What is there to be gained from other languages? Should you learn another language to make you a better programmer? Should Python incorporate your favourite feature from another language? Will that make Python a better language? Why does Python have lambda expressions? Generator expressions? What other languages have them? David will present some material that highlights the common ideas shared between many languages (some more dynamic that others), but more importantly allows each language's differences to be contrasted, thereby giving insight into that language's philosophy.
Attendees will be expected to have opinions, and to voice them!
Discover connexions with Fortran, Haskell, Pascal, Lua and ML, and review the more patent connexions with the usual suspects: C, Java, JavaScript.
GIL isn't evil
Parallel programming using Python
Russel Winder (Concertant LLP)
Talk, 60 minutes (intermediate)
Every computer is a parallel one these days. Every program really needs to harness parallelism if it is to perform well. Python has the (notorious) GIL (Global Interpreter Lock) which many people think means that Python cannot be used to write parallel programs. Parallel Python and the multiprocessing package show that this is not the case. So GIL isn't (necessarily) your enemy, and CSP may actually be your friend.
The Clean Code Challenge
What does clean code look like in Python?
Talk, 90 minutes (general)
Can you write really good, clean code? Or does your Python usually look like slightly mangled Java?
Bob Martin recently published a book called “clean code” where he sets out what he thinks good, clean code looks like, "in hideous detail" as he puts it. One of his ideas is to encourage programmers to write better code, by providing worked examples of good coding practice that they can follow. All the examples are in Java, which is something of a limitation, but one chapter of the book piqued my interest with a little footnote: “I recenly rewrote this module in <a dynamic language>. It was 1/7th the size and had a subtly better structure”. From this I infer that what “clean code” looks like must be language-specific. So what does clean code look like in python then? How does it compare to Bob Martin's Java?
At this session I would like to do some coding to try to answer this question. I have taken a little Java program called “Args”, which does some simple command line argument processing. I translated it directly into python, preserving as much of Bob Martin's original Java design as possible. I have to say, it looks pretty far from “clean” python to me. What I want to do at this session is for us to split into small groups and refactor this code until it looks clean and pythonic. Will it turn out to be 1/7th the size and have a subtly better structure? Come along and have a go!
Session outline: 15 minutes – introductory presentation 60 minutes – split into pairs/small groups and get coding 15 minutes – debrief/feedback
We'll post all the code online before and after the session.
Decorator and Metaclass Metaprogramming
Introducing Metaprogramming with both decorators and metaclasses
Bruce Eckel (MindView, Inc)
Talk, 60 minutes (intermediate)
The goal of metaprogramming is to provide greater leverage by writing code that modifies other code. This presentation introduces metaprogramming and shows examples using both decorators and metaclasses. Although decorators are preferable because they are more straightforward, there are situations where metaclasses are still necessary; we'll look at these special cases and visit some initialization issues along the way.
Ctypes Tips and Techniques
Avoiding potential ctypes pitfalls
Talk, 45 minutes (intermediate)
ctypes is a very powerful package for integrating C-language libraries into an existing Python install. In this talk, we will discuss the ctypes infrastructure and how to avoid some potential pitfalls.
Outline:
- Quick ctypes overview
- Why ctypes?
- Function prototypes
- System functions
- User-written functions
- Adding callbacks
How does ctypes work?
Object / structure ownership
Garbage collection
Static fields & function pointers
- Callbacks
- Function prototypes
- Return types
- Threads
- GIL consequences
- Thread-local storage
Loving Old Versions of Python
See the benefits of standing well back from the bleeding edge.
David Jones (Ravenbrook Limited)
Talk, 30 minutes (beginners)
With Python 3.0 released and Python 3.1 just around the corner, I'd like to invite you to take a step back. Step out of the fast lane. Relax, you don't need the latest greatest feature to complete your winning killer app. Why would we want to write our programs in older versions of Python? Well, portability.
I recently wrote a program originally targeting Python 2.4. Then I "ported" it to Python 2.3, and then to Python 2.2. In this brief talk I explain why you might want to use older versions of Python; why I did, and what problems I encountered.
Python Foundation Class
A one day Python Tutorial concentrating on how it works 'under the hood'
John Pinner (Clockwork Software Systems)
Nick Booker (Clockwork Software Systems)
Tutorial, Full Day (beginners)
If you are a programmer new to Python, or if you have 'picked it up', this tutorial will give you a solid foundation for the future.
The tutorial will start by getting to know the Python interpreter and taking a look at Python's introspective features, moving onto data types, sequences and dictionaries. This will be followed by functions, classes and modules and by the end of the day you will know how to develop your Python knowledge further.
Bring your laptop and get to know the Python interpreter! If you haven't got a laptop, let us know and we may be able to arrange a laptop loan (no promises).
Code Clinic
Bring your code for constructive peer review
David Jones (Ravenbrook Limited)
Michael Foord (Resolver Systems)
John Pinner (Clockwork Software Systems)
Panel, Half day (intermediate)
Have you got problems with some Python coding? Would you like some help from your peers and our resident experts? Or have you got some ideas you'd like to show off? Bring your own code and have the group go over it and make suggestions for speed, clarity, technique, etc.
This is a really useful and enjoyable activity.
If you want to know more, look at the wiki http://wiki.europython.eu/CodeClinic/, you can even sign up there!
Multimedia
Python and Nuke
Use of Python in film and post-production software.
Nuke is a powerful video compositing application that delivers unparalleled speed, an efficient multi-channel scanline rendering engine, and a first-class feature set that is unrivalled in the desktop market. The use of Python in Nuke allowed us to provide an interface suitable to integration within film pipelines within studios worldwide.
Goncalo Carvalho (The Foundry)
Talk, 30 minutes (intermediate)
The talk discusses the design decisions and provides examples that describe how an application built around TCL was successfully converted into Python (as a scripting language) while maintaining backwards compatibility and at the same time enhancing its appeal and lifetime.
gSculpt 3d Modeller
The inner workings of a procedural 3D modeller written in Python and C++
Talk, 45 minutes (intermediate)
gSculpt is a procedural 3D modeler developed in Python and C++. In this talk, I intend to discuss some of the issues involved in developing procedural software, and linking Python to C++. I will also discuss some of the techniques used to develop gSculpts data model and UI system.
Network
Twisted, AMQP and Thrift
Bridging messaging and RPC for building scalable distributed applications
Talk, 45 minutes (advanced)
Summary
The messaging/RPC architecture I will describe is a core component of FluidDB, a new online database being developed by Fluidinfo. We built Twisted-friendly [1] interfaces to both Thrift [2] and AMQP [3]. The combination of Thrift, AMQP, and Twisted is extremely powerful and flexible.
AMQP was developed in the context of the high-stakes high-throughput financial world. It has proven performance, is an open standard, and has several open-source implementations. We first built a Twisted class for AMQP clients, and then added a Twisted-capable version of Thrift to support passing Python objects, exceptions and Twisted Deferreds during RPC. We have open-sourced both components [4] and [5].
It is increasingly expected that modern applications will scale to handle heavy loads. This implies services that use multiple servers, which has in turn led to a growing realization that in many cases an important part of the solution should be a reliable messaging architecture. To stay competitive, and to argue for Python in development and deployment, Python application programmers will increasingly be in need of architectural components that allow them to build such systems.
For these reasons, we believe that our Thrift-AMQP-Twisted approach can play an important role in future Python projects comprising multiple communicating services that need to scale.
Talk Outline
- Intro - aims, requirements, options considered (5 mins).
- Intro to AMQP (3 mins).
- Intro to Thrift (3 mins).
- Intro to Twisted (1 min).
- Example code, including raising exceptions and returning Deferreds (10 mins).
- Benchmarks (3 mins).
- Conclusion: Why you might want to use this code in your project (6 mins).
- It's fast in operation and simple to work with.
- It provides a good basis for building scalable applications.
- It gives you a good leg up on building reliable replicated services.
- You can use it to do load balancing between internal services.
- Allows transactions, including distributed transactions.
Links
[1] | http://www.twistedmatrix.com |
[2] | http://wiki.apache.org/thrift |
[3] | http://amqp.org |
[4] | http://archives.free.net.ph/message/20080929.203728.84bb862a.en.html |
[5] | https://launchpad.net/txamqp |
Castles in the Cloud
Software architectures to exploit the rich cloud-computing resources of Amazon & Google.
Kevin Noonan (Calbane Ltd)
Talk, 60 minutes (advanced)
We are in the heat of what Bill Gates dubbed the "services wave" of the digital revolution. Computing resources as fundamental as processors and storage have been abstracted and provided as services from remote data centers.
2009 is the year that cloud-computing begins its move to mainstream, with the launch of Google App Engine as a commercial offering and a dramatic price drop by Amazon for long-lived EC2 instances.
Perhaps you know the basics of cloud-computing? Amazon S3 is a virtual file-store, EC2 is a virtual server, Google App Engine is a web-stack.
Do you know that App Engine features the first public API for BigTable, Google's planetary datastore, which performs successfully at petabyte scale across diverse applications from search to finance to Google Earth?
This talk will go beyond the basics to explore the many ways to slice-and-dice these new platforms from Amazon & Google.
- Use BigTable as a remote database (or backup) for desktop apps--or for web-apps hosted elsewhere.
- Use EC2 as a demand-overflow server, starting it automatically as your primary server reaches a load-watermark.
- Exploit new scheduled tasks and work queues to move App Engine beyond a web-stack to a general computing platform.
- Freely run open-source software such as a blog or CMS for your project or company on App Engine.
- Use App Engine or Amazon S3 to serve static web-content (for a site hosted elsewhere).
- Use App Engine to host web-services with an XML-RPC or JSON-RPC interface.
- Use Amazon SQS as a work queue for an EC2 server, an alternative task-queue for App Engine, or for any other distributed application on the planet.
The foregoing scenarios will be illustrated with architecture diagrams and code-samples.
Science
Cluster Programming with PIPPER
Python Integrated Parallel Programming EnviRonment (PIPPER) automates cluster parallelization of Python code. It is a Python preparser that uses programmer directives, similar to OpenMP, to create a parallel Python script that runs across several computers on a cluster.
This talk will introduce the core concepts of PIPPER, and the requirements for creating a working script.
http://pipper.sourceforge.net/
Kyle Ellrott (UCSD)
Talk, 60 minutes (advanced)
- Introduction a) Introduction to trivially parallelizable calculations (5 min) b) The difference between Symmetric multiprocessing (SMP) and Asymmetric multiprocessing (ASMP) (1-2 min) c) Why object serialization ('Pickle') is important ( 4 min ) d) OpenMP, MPI, ( and queue spaming ) ( 4 min )
- Writing for PIPPER a) 'Hello World' (4 min) b) Constraints of the PIPPER syntax (6 min) c) Pipper __name__ values (5 min) d) Using Iterators (3 min) e) Things to avoid (5 min)
- Final thoughts a) How PIPPER compares to a method like Map-Reduce (5 min) b) Ways that PIPPER could be improved (5 min)
Python Climate Data Publishing
This talk is about a Modular Climate Data Staging Framework for the C3-Grid in Python. It simplifies for the climate scientist the implementation of service functionality for feeding scientific workflows with input data from large heterogeneous distributed climate data stores using Python and the data modification tools he/she knows well through dealing with concurrency, XML metadata updates, result transmission, runtime estimations for scheduling and so on. On the way, the audience will have a little insight in scientific workflows, abstraction approaches using meta data and useful tools for work with large (climate) data sets.
Henning Bergmeyer (German Aerospace Centre)
Talk, 30 minutes (general)
C3-Grid is a grid infrastructure project for the German Climate Community that ended in Feb 2009. Users access large scale, distributed, heterogeneous data and processing workflows over a grid portal. The uniform binding of data repositories into the grid via services requires abstraction of the different storage systems, data formats, data structures and naming conventions. The talk is about a Python framework compatible with Python >=2.4 that eases the modular development and re-use of so called staging scripts that steer the data processing command line tools and libraries well known by the data providers according to the standardized request parameters and ISO metadata used in C3-Grid. The talk stresses the fact that most data providers are scientific users from the climate community and not computer scientists. The stager development has to be easy, guided and focused on clear data centric necessities, while hiding infrastructure related issues as well as possible.
Breakdown: The talk briefly introduces basic terms from scientific data processing and the grid such as workflows, staging and metadata. Then the C3-Grid infrastructure regarding to data delivery is explained and in what way its strength in terms of data store abstraction can lead to error-prone service implementations, when not carefully carried out by computer scientist. The main part of the talk depicts how the library takes over a lot of the repetitive tasks and provides a simple, clean interface to realize data set specific steps. As examples two implementations of data preparation methods are shown: One integrating a command line tool (cdo) for working on NetCDF files and another using the PyHDF library with its brilliant array extraction features from NumPy.
Using the Modular Toolkit for Data Processing (MDP) Library
How the Modular Toolkit for Data Processing (MDP) can help you.
Talk, 30 minutes (intermediate)
This talk will present the open source MDP library and demonstrate how to use it for data processing. It will be shown how easily one can apply the included algorithms, combine them and implement new ones. Typical applications are the extraction of relevant signals from high-dimensional data, e.g. time series.
MDP originated in theoretical neuroscience, but is designed to be useful in any context where trainable data processing algorithms are used. From the user's perspective, MDP is a collection of supervised and unsupervised learning algorithms and other data processing units that are encapsulated in nodes with a standardized interface. The nodes can be combined into data processing sequences and more complex network architectures. Some included algorithms are for example Principal Component Analysis (PCA, NIPALS), several Independent Component Analysis algorithms (CuBICA, FastICA, TDSEP, and JADE), Restricted Boltzmann Machine, and Gaussian Classifiers. Users can extend MDP by creating custom nodes to take advantage of the provided framework. MDP already includes several nodes that have been contributed by users (e.g. Locally Linear Embedding).
The talk will also briefly introduce some new MDP features, like convenient parallelization, hierarchical networks and bidirectional data flows.
Testing
DSL for Functional Testing
Domain Specific Language for Functional Testing
Mark Fink (Tabane Limited, Schmiechen, Germany)
Talk, 45 minutes (intermediate)
I describe an approach on how to successfully implement functional test automation which significantly reduces cost of ownership of a test suite in comparison to the traditional approach. My approach mitigates the risk of fast evolving software through the use of maintainable functional test automation.
My contributions:
- I will propose an approach that will significantly reduce the total cost of ownership of an automated test suite.
- I have been experiencing in practice that domain experts are not involved in testing and what is worse domain specific tests are neither documented nor executed. I show how this approach involves the domain experts in the functional test automation.
- I analysed existing approaches that could support DSL Modeling. In the talk I show how to use the existing knowledge and leverage it in order to successfully jumpstart a test automation initiative.
- I will demonstrate how to implement this approach in Python.
- I will use an up-to-date example throughout the whole talk to show the utilization of the various aspects of the approach.
Timeline for the talk
The Problem (10 min)
It is still not clear if and in what way functional test automation can significantly contribute to software quality. Success or failure of service oriented companies heavily depends on the quality of their software.
- Automated functional tests are expensive and difficult to maintain
- Capture an Replay Method Demonstration using Selenium IDE for capture and replay
- Domain Specific Tests are not executed
The Idea (5 min)
Use a Functional Testing DSL
DSL Modeling (10 min)
- Domain Specific Language Definition
- DSL Design Goals
- Feature Diagram
DSL Implementation (12 min)
- Software Layers
- Internal DSL vs. External DSL
- Method Chaining
- Implementat of a Internal DSL in Python
- Functional Testing DSL (Code walkthrough + Demonstration)
Conclusions (3 min)
Trade complexity within DSL framework with ease of test writing!
Questions and Answers (5 min)
Using Windmill
This talk will get you writing and running automated tests and show off some of the most useful buil
Talk, 30 minutes (beginners)
Using Windmill Intro (3 Minutes) • Brief History • Project Goals • Project State • Talk Agenda Windmill has deep open source roots, and has grown organically based on the needs of users.
Installing (2 Minutes) • Supported Platforms • Requirements • Easiest method of installation • Developing and contributing
Windmill IDE (10 Minutes) • Supported Browsers • Controller API • Tools (DOM Explorer, Firebug, Firebug Lite) • Building, recording and editing tests (Assertion Explorer) • Playing and debugging tests in the IDE • Generating and saving test files • Results and performance output Windmill Service (10 Minutes) • Python Tests • JavaScript Tests • Running and debugging your tests from the command line • Test dependencies and hierarchy • The shell environment Continuous Integration (3 Minutes) • Features • Setup/Teardown • Reporting Q & A (Remaining Time)
Acceptance Testing with RobotFramework
Acceptance testing and acceptance test-driven development (ATDD) using a Python based test automation framework. This is an introduction build around demonstrations and requires no earlier experience.
Pekka Klärck (Eliga Oy)
Talk, 60 minutes (beginners)
Robot Framework (http://robotframework.org) is a generic open source test automation framework for acceptance testing and ATDD. It used the keyword-driven test automation approach and has an easy-to-use tabular syntax for creating test cases. Its testing capabilities can be extended by test libraries implemented with Python or Java (through Jython).
This introduction goes through the basic concept and most important features of the framework. Most of the session is reserved for demonstrations where the framework is used for web, GUI, and API testing using different testing techniques. The main topics covered are:
- The high-level architecture and the difference between frameworks and drivers.
- Creating test cases using the keyword-driven approach.
- Keywords with different abstraction level and data-driven tests.
- A short introduction to ATTD with examples.
- Creating new test libraries with Python.
- Available test libraries and other supporting tools.
Text-based Acceptance Testing Using TextTest
This session aims to present a technique and a tool for automating maintainable black-box tests, that works entirely outside of the code. TextTest calls the system under test via the command line and compares plain-text output against previously accepted versions of that output.
Talk, 45 minutes (general)
Testing a program by comparing text files written by the current run with those from previous runs is an old idea, but one which has fallen out of fashion. This talk aims to rehabilitate it as an agile technique and present a suitable tool, TextTest. We aim to show that there are applications where testing in this way has major advantages, and examine the advantages and disadvantages in a more general setting.
TextTest is a standlone tool and an extensible Python framework, and while it can test programs written in any language, it is especially suited to the dynamic nature of Python and to testing small command-line scripts where Python is also at home. There will also be a brief practical demonstration.
This talk is aimed at technical people of any experience level who are interested in techniques for testing their programs in a more black-box way than offered by doctest or unittest, i.e. specifying tests in terms of how to run the program and what to expect rather than in terms of the code itself.
PyUseCase: Testing PyGTK GUIs
PyUseCase is a record/playback GUI-testing tool for PyGTK GUIs. But it isn't like other record/playback tools. Instead of recording a series of "enter text", "click button", it aims to record a "use-case", or a high-level description of the intent behind those actions.
Talk, 45 minutes (intermediate)
There are two main approaches to testing a UI: 'record-playback' through the user interface and 'data-driven' bypassing it. Testing methods which use the former approach are visual and user-friendly but have a reputation for being unmaintainable, so the world has largely embraced the latter. But could we recover the advantages of record-playback without sacrificing maintainibility? This would mean moving beyond recording screen mechanics or GUI toolkit mechanics and actually trying to record the intent behind them. The approach presented achieves this via manually maintaining a mapping of UI actions to domain language statements.
An actual library, PyUseCase is also presented and briefly demonstrated. PyUseCase consists of a core framework that could be extended for any rich-client Python UI (e.g. PyQT, wxPython), and a PyGTK-specific part that handles the most common PyGTK widgets. So far it has only been extended when support for new widgets has been needed: no attempt has been made to exhaustively cover the widget set. The same approach could lead gradually to coverage of other GUI toolkits.
Hands-on testing with TextTest
This session aims to present a technique and a tool for automating maintainable black-box tests, that works entirely outside of the code. TextTest calls the system under test via the command line and compares plain-text output against previously accepted versions of that output. In this session you will install it on your laptop and try to automate tests for various toy programs.
Tutorial, Half day (general)
Testing a program by comparing text files written by the current run with those from previous runs is an old idea, but one which has fallen out of fashion. This talk aims to rehabilitate it as an agile technique and present a suitable tool, TextTest. We aim to show that there are applications where testing in this way has major advantages, and examine the advantages and disadvantages in a more general setting.
TextTest is a standlone tool and an extensible Python framework, and while it can test programs written in any language, it is especially suited to the dynamic nature of Python and to testing small command-line scripts where Python is also at home.
We will present the tool for the first half-hour or so and then get people to install it on their laptops and help them through a series of toy programs to test with it. (Starting with Hello World and getting progressively more complicated). We will cover testing scripts, legacy systems and possibly PyGTK UIs depending on how well people get on. We will finish with a "retrospective" where there will be an open discussion on what we have learned.
Attendees should be technically capabale and reasonably familiar with Python. They should bring a laptop, with Python/PyGTK installed following the instructions given at http://www.texttest.org/index.php?page=documentation_trunk&n=install_texttest .
Rapid testing with py.test
This tutorial provides in-depth information about writing tests in Python and running them with py.test.
Holger Krekel (MerLinux GMBH)
Tutorial, Half day (intermediate)
Want to dive deeper into automated testing with Python? Want to know how to rapidly write all kinds of tests and organise your test suite with minimal boilerplate? Want to use a tool that allows you to ad-hoc distribute tests to multiple machines and platforms?
Prerequisites/knowledge: good knowledge of Python programming, basic familiarity with automated testing Requirements: Attendees should bring their laptops with Python installed (version 2.4 or higher).
Synopsis
We walk through basic consecutive examples, the current feature set and exercise ways to setup and configure custom test runs. In the second part we distribute tests to multiple CPUs and platforms and discuss the basics of extension mechanism.
The tutorial format will be an interactive lecture with some exercises and plenty of time for questions. Please bring your laptop and have Python installed and working.
Tutorial Outline
Intro: quick round on backgrounds of participants and talker
Terminology/Overview (20 minutes) - developer and "customer" tests - small/medium/big tests aka unit/functional/integration - acceptance tests - benefits of automated testing
Walkthrough Python test functions (60 minutes) - installing, basic usage of py.test - test functions - test classes - interactive debugging and pdb introspection - using setup state (fixtures) in your tests - adding a command line option for test setup - parametrizing tests with multiple values - looponfailing: run test set until all tests pass
- break *
distributing tests (30 minutes)
- run your tests on multiple CPUs/multicore
- running tests on multiple machines at once
- running tests on windows, driven by Unix
- do's and dont's for cross-process testing
testing extensions (20 minutes) - traditional unit tests - doctests - ReST documents - Javascript tests
- break *
Writing extensions (30 minutes) - hook mechanism - local and global plugins - test collection hooks + example - test running hooks + example
Buffer, Q&A (15 minutes)
Resources
project: http://pytest.org holger: http://tetamap.wordpress.com
Testing in Python with py.test
This talk introduces the 1.0 relase of the popular test runner py.test.
Holger Krekel (MerLinux GMBH)
Talk, 60 minutes (intermediate)
This talk introduces py.test a a general test runner. There are many exciting new features with the 1.0 release. I'll walk through some highlights, notably the plug-in system and methods for load-balancing long-running tests and performing simultaneous cross-platform test runs. I'll also give an overview of existing plug-ins, including ones for coverage testing, unittests, doctests and javascript-tests and provide an outlook of what's next.
Testing Javascript with py.test
JavaScript testing as first-class citizen with py.test
Samuele Pedroni (OpenEnd AB)
Talk, 30 minutes (intermediate)
Web applications often need quite a bit of JavaScript code to increase responsiveness and interactivity of pages, so called AJAX techniques.
This JavaScript code merits and needs to be tested as much as the server code.
Functional testing is one possibility for it with tools like Windmill, but it can be too cumbersome or at the wrong level for complex code.
At Open End we have developed support for py.test to be able to drive and run unit tests for JavaScript code by executing them in the actual browsers. Running integration tests with the server code with as much faking as needed is also possible.
This support is being open sourced and the talk will describe it and its use.
Unittest: It ain't broke; lets fix it!
Why unittest is better than you think
Jonathan Lange (Canonical)
Talk, 45 minutes (general)
unittest first appeared in Python 2.1 as a port from the popular JUnit testing framework. Python has changed a lot since then, and the Python community has learned much about testing and what it means to write Pythonic code.
Should we write a new testing framework, a purely Pythonic one free of any Java (or Smalltalk) heritage? The authors humbly suggest "no": unittest is very powerful and extensible, it's just not quite extensible enough.
We present some stories of how unittest has been used by major projects, how it has been abused (often by us) and the consequences of that abuse. We look at how these disasters could have been avoided.
Web Programming
Python and CouchDB
Introduces new users to CouchDB and covers a variety of methods for using CouchDB in Python
Talk, 30 minutes (intermediate)
Introduction to CouchDB - Documents (JSON, uuids, revisions) - REST API (simple http api for access) - Views (design docs, http api, javascript view server, alternate view server) - Eventual Consistency (distributed storage, replication) - Futon (web interface to couch, different relationship between you and your DB) CouchDB in Python - Raw Access (httplib2 and simplejson) - Client Libraries (couchdb-python, pouch, couchapp) Common Patterns - Differences between a schemas and restrictions - CouchDB with Django - Handling changes to data over time Q & A
Reliable Python Web Sites
Tips, tricks, and strategies for making your Python web sites reliable and stress-free
Andreas Kaempf (Soundience Limited)
Talk, 30 minutes (intermediate)
Python, with its rich libraries, is an ideal platform for building web sites with rich functionality. Because these web sites are often composed of various components, however, achieving high reliability and minimising down-time require careful planning and attention. This presentation describes some high level strategies, as well as detailed tips, tricks, and battle stories, for achieving high reliability, automating as much as possible to reduce manual intervention when things go wrong.
TurboGears: a framework reborn
Why we rewrote TurboGears and how it can help you write web applications
Mark Ramm (SourceForge)
Talk, 30 minutes (general)
TurboGears 2 is a largely API compatable rewrite of the TurboGears framework, but it's also a revision of the whole idea of TurboGears.
The rewrite is over, and TurboGears 2 has all kinds of new features to help web application developers get stuff done. This talk will discussion the current state of TG and will outline some of our plans for the future.
The Zen of Pyjamas
How Pyjamas blends desktop and web development into One...
Talk, 90 minutes (general)
Pyjamas is a refreshing and radically different approach to AJAX programming: it allows the programmer to think, design and code web applications entirely in Python, yet the applications will still actually run natively in every single major web browser (no plugins required: only javascript).
Pyjamas is a python-to-javascript compiler and widget set that effectively turns the web browser into a desktop widget platform, thanks to the DOM model javascript bindings.
By manipulating the DOM model direct, Pyjamas startlingly puts web browser technology on a par with Python-GTK2 and Python-Qt4, with the added advantage over-and-above these "native" desktop widget sets of having complete and full access to SVG 2D Canvas, native javascript, HTML, CSS and NPAPI plugins such as Flash.
The Pyjamas User-Interface widget API is in fact so startlingly similar to Python-GTK2 and Python-Qt4 that two sub-projects have been done: one is to port the GTK2 API on top of the Pyjamas API; the other is to port the Pyjamas API onto the desktop, using Webkit, bypassing and cutting out the javascript entirely and calling into the DOM model direct, from python gobject bindings to Webkit's DOM model.
The talk will go over what Pyjamas is, and what it does, and the talk itself will actually be run as both a Web Application and a stand-alone Desktop application running from the same identical Python source code.
Python to Javascript and Back
Compiling and using Javascript as a JIT Optimising Compiler (PyV8)
Talk, 30 minutes (advanced)
This talk is part of a series on the Pyjamas Project: it will specifically cover the PyJS compiler in technical detail, and will cover the two experiments that are underway to chain Python to Javascript to Google's V8 JIT optimised x86 / ARM assembly Javascript compiler.
Simple tests such as running a fibonacci algorithm by converting it to javascript and handing it to PyV8 show a dramatic 10-fold speed improvement when compared to running the same simple algorithm using the standard python interpreter.
Also, thanks to PyV8 and the design of V8, access to standard c-based python modules is still possible.
The limitations of javascript (which aren't many) as an intermediate compiler target, and how the limitations may be overcome, will be discussed.
Pyjamas and Django
How to best use Pyjamas and Django to develop Web apps
Talk, 30 minutes (beginners)
This is part of the Pyjamas series of talks, with this talk specifically covering the best ways in which web developers can get the most out of the Pyjamas and Django combination.
Early developments with Pyjamas and Django were simply to add a 20-line JSONRPCService class on the Django server-side, and to use the Pyjamas JSONProxy client-side RPC class to make function calls in a near-transparent fashion.
Recent developments in Django include experiments to superclass the JSONRPCService and combine it with Django Forms! Thus, the server-side logic expressed in standard Django forms can be accessed with a simple one-line function call on the client-side, inside the web application.
Work is underway at the time of writing to extend the DjangoFormJSONservice class to include a "save" method as well as a "validate" method.
Future work which may possibly be completed by EuroPython may include the generation, server-side, of JavaScript Pyjamas classes auto-generated from Django Forms (!) This will involve auto-generating small Python/Pyjamas code fragments that will then be handed to the PyJS compiler by Django. So instead of a dynamically-generated HTML Form being created by Django, dynamically-generated Javascript will be created that does exactly the same job as the HTML version.
Mashing up The Guardian
How to get access to guardian news content in Python.
Michael Brunton-Spall (Guardian News and Media)
Talk, 60 minutes (beginners)
The Guardian, Europe's largest online newspaper, recently announced the Guardian Open Platform (http://www.guardian.co.uk/open-platform), allowing developers access to over 10 years worth of Guardian editorial content. This talk will take you on a whistlestop tour of the open platform, showing you how to apply for and use our API keys, how the API itself works, and sorts of things have already been done with our data.
I'll also be showing you our open source Python library for accessing Guardian content, and giving you an example of how to integrate it into your own Google App Engine or Django website
Nevow and Stan
How to build websites with Twisted packages Nevow and Stan
Tutorial, Half day (beginners)
Building websites using the Nevow web framework and the Stan DOM gives you full control, and allows you to work with HTML templates and Python code neatly separated. It also allows you to build very nicely scalable websites with database integration and other backend functionality.
Unfortunately the documentation of Nevow is not very beginner friendly. This tutorial is intended to break the initial resistance and get you started in building working web applications.
You will come away from the tutorial having written a small website with forms, authentication and SQL integration. You will also have a library of useful patterns for your own website construction.
You should have Python 2.5, Twisted (8.1 or later), Nevow (0.9.32 or later) and Sqlite (3.6 or later) installed on your laptop for the tutorial. (Python 2.6 will probably work too, as the latest version of Nevow passes its test suite under 2.6, however, it has not been extensively tried in deployed applications). A package of files will be available over the Internet and on a USB stick for you to install at the outset of the tutorial.
For the tutorial you need basic knowledge about how to program in Python and a basic understanding of HTML.
Semantic Apps with CubicWeb
A framework that makes building semantic web apps a construction game.
Talk, 45 minutes (intermediate)
The CubicWeb framework was designed to develop semantic web applications that have both a HTML/Ajax rich user interface and a RDF/OWL-based data interface.
We will compare the design of Cubic Web with other web frameworks, focusing on key differences lsuch as applying views to selections of objects implementing the required interface, multi-database support, migration and versioning for maintenance over several years, and truly reusable components.
We will present examples of integration within the Linked Data cloud, and with Google Maps, and also present of faceted search.
Zope Page Templates
An introduction to Zope Page Templates and their use outside of Zope
Matt Hamilton (Netsight Internet Solutions)
Talk, 30 minutes (beginners)
Zope Page Templates have been around for a while, and used extensively in Zope and many Zope based apps and frameworks, but did you know you can use ZPT with any Python project? Indeed there are implementations of the syntax used, Template Attribute Language (TAL), for other languages too. Making it one of the most portable, cross platform templating languages there is. Find out why ZPT and TAL are so elegant, and how to use them with your Python project.
I will cover why TAL is a great choice for templating, the simple syntax of TAL and how to create and render page template objects in your code.
The Zen of Web
Python has an overwhelming number of Web frameworks, but unfortunately they don’t help us understand the dos and don’ts of Web publishing. This talk will cover the basics that every developer should know about the Web and some concrete examples of how to follow these best practices with Django.
Talk, 45 minutes (beginners)
The secret of Web publishing is that it's stupidly easy. But it's also really hard to do well. This talk is intended for Python programmers who use (or want to use) Web frameworks like Django or Pylons but are not familiar with Web basics like the difference between POST and GET.
This talk will cover things like basics of Web accessibility, usability, and security, designing good URLs, when to use POST and GET, and, also some new hype words like Ajax.
The examples given in this talk will be written for Django, but they are not Django-specific in any way.
Tapping into the Web of Data
Learn how to SuRF structured data from the Web using Python
Talk, 45 minutes (intermediate)
There are vast amounts of data available on the Web, made available by the community Linked Open Data project and many online community sites (using data formats such as FOAF, SIOC and DOAP). Similarly as the Web already allows documents to link to each other, linked data refers to linked, machine-readable data sets on the Web.
The Open Linked Data project is a community effort to make existing, well-known data sets like Wikipedia, Geonames or Musicbrainz available as RDF data on the Web and ready for reuse. Developers get a unified interface to data from many sources instead of one API for each site.
In order to take advantage of this, there is a need for easy-to-use tools that allow developers to work with this data without knowing the specifics of RDF, SPARQL or other related Semantic Web technologies.
In this talk, we will outline the SuRF library, that provides Python developers a way to build Semantic Web applications using RDF data in an easy way. We will show how it can be used to provide advanced browsing interfaces and mash-ups services on the top of existing data, and how it can be efficiently combined with other frameworks such as Pylons for Web-based applications.
SuRF is an open-source "Object-RDF Mapper", similar to object-relational mappers (e.g., SQLAlchemy). One can access SuRF objects and use their properties through a virtual API generated on the fly by reflecting the underlying RDF data. SuRF is built around a plugin architecture with built-in support for the most common Semantic Web data access protocols and API's.
Talk outline: (1) Introduction; (2) Architecture of SuRF; (3) Usage of SuRF; (4) Advanced applications (e.g. integration with Pylons)
Lipstick on a Pig
Dynamically skinning a legacy portal using Python, WSGI (the Python Web Server Gateway Interface), and Deliverance.
Matt Hamilton (Netsight Internet Solutions)
Talk, 45 minutes (general)
So you have a big legacy portal application which you want to change the look of, but are contractually not allowed to touch?
Here is a case study on how we used the power and flexibility of Python and WSGI and the wonder lxml to dynamically re-skin a proprietary .NET portal without even touching it.
We take a giant lump of messy invalid HTML markup and dynamically strip it back, add semantic markup and CSS and present the user with a nice svelte valid site.
I will cover the history of the legacy portal, the problems encountered, our cunning plan to dynamically re-skin the site, a technical overview of the parts of the system (lxml, WSGI, etc), and what we learned along the way.
Web Frameworks and Jython
A look at popular Python web frameworks that are now working on Jython
Frank Wierzbicki (Sun Microsystems)
Talk, 45 minutes (intermediate)
A final release of Jython 2.5 should be out by the time of this presentation. The Jython project has been collaborating with a number of web frameworks to make sure that they work well on the Jython platform. At the time that this abstract was written, Django and Pylons are working well on Jython, and there has been good progress on getting TurboGears to run. In this talk, I will demonstrate how to take advantage of Java-based technologies from these web frameworks, including the use of Java libraries and deployment options. Some knowledge of Python web frameworks is assumed, but no knowledge of Java is required.
Real world App Engine Projects
How to get the best out of App Engine, and related technologies such as Bigtable, Amazon S3, EC2 and Cloudfront.
The audience for this talk should know the basics of Python the programming language, and of the web.
Talk, 45 minutes (intermediate)
This talk covers the use of Google App Engine for real world portals that go beyond a small number of users or content objects. It is shown how Lovely Systems integrates different technologies to meet the requirements of such portals. Technologies used are:
- The App Engine Python SDK along with Google's Bigtable to store small objects
- Amazon S3 to store files
- Pyjamas to generate the Javascript based backoffice interface communicating via JSON-RPC
- Amazon EC2 for tasks which are not possible on App Engine
- Amazon's Cloudfront for delivering static content
XML Document Publishing
Publishing XML Documents with Python, Django and Berkeley DB XML
Zeth (University of Birmingham)
Talk, 45 minutes (intermediate)
I will be talking about a new approach for publishing large complex XML documents and document collections. I work in the context of academic electronic publication, however this approach can be used for publishing any data encoded in XML.
The 'Pixelise' module was first discussed at PyConUK 2008, and since then it has been used in production. The default setup is to use the Pixelise module with Django and with the native XML database Oracle Berkeley DB XML. This provides Django with the functionality to create/update/query the native XML database, and more importantly provides Django with the ability to generate dynamic content according to the structure of the document contained in the native XML database, or even just a specific section of a document.
So if someone gives you a pile of XML to publish, then this is the talk for you.
JavaScript for Python users
Help for Python programmers who have to write good JavaScript
Tutorial, Half day (general)
The talk will cover the fundamental language features of JavaScript, and will compare JavaScript and Python.
- It will cover and explain JavaScript gotchas, such as
- tmp = b.c; tmp()
- not being the same as
- b.c();
as well as basics such as inheritance.
Silva Roundtable
Silva users face-to-face: meet the people behind the names on the Silva channels
Kit Blake (Infrae)
Meeting, 45 minutes (general)
Users BoF session to discuss the Silva roadmap, new features, wishlist, problems, and organizational needs. Look for collaboration opportunities, setup smaller meetings for later during the conference. Share ideas, plans, and code.
Crowdsourcing with Django
How the Guardian built a Django app to crowdsource MP's expenses analysis in 4 and a half days.
Simon Willison (Guardian News and Media)
Talk, 45 minutes (general)
The Guardian recently released http://mps-expenses.guardian.co.uk/, a crowdsourcing application inviting members of the public to help dig through the 400,000 pages of MP expenses documents released by parliament. The application was developed in Django in just four and a half days and was deployed using EC2, both of which were firsts for the Guardian. This talk will cover lessons learnt building and scaling the application, as well as larger lessons learnt about crowdsourcing in general.