Aller au contenu | Aller au menu | Aller à la recherche

/var/log/iksaif

samedi, décembre 3 2011

celery and django-celery ebuilds for Gentoo

I'm currently working on a secret project using django and celery, and I needed ebuilds for celery since there are none in official portage tree. I started with some ebuilds found on quantumsummers overlay, but they were really outdated.

So I wrote some new ebuilds for celery, djang-celery and related dependencies. I only tested them on my server but I think they should work for you. They are available on my overlay: http://git.iksaif.net/?p=portage.gi...

samsung-laptop:

I recently got a Samsung Series 9 from Intel. The laptop is really great, but some features like keyboard backlight where not working correctly. The driver responsible for that is samsung-laptop (drivers/platform/x86/samsung-laptop.c).

So, having some experience about x86 platform drivers, I took a shoot and tried to hack the driver to make it work. This driver does not use ACPI or WMI but poke directly the BIOS instead using an interface called SABI (Samsung ABI ?). So the hard part was to understand how this interface work, and of course, without any kind of documentation. I may describe later how I did it, but it was fun :).

So, if you have a samsung laptop, you should try the new samsung-laptop driver ! The code is not yet merged in mainline, but its available on github:

The code was tested on 3 different models, and I'm looking for more feedback.

vendredi, avril 15 2011

euscan: how to find new upstream versions ?

What is euscan ?

dev-portage/euscan

euscan is available in portage as a dev package (app-portage/euscan-9999). This tool allow to check if a given package/ebuild has new upstream versions or not. It will use different heuristic to scan upstream and grab new versions and related urls.

This tool was designed to mimic debian's uscan, but there is a major difference between the two: uscan uses a specific "watch" file that describe how it should scan packages, while euscan uses only what can already be found in ebuilds. Of course, we could later add some informations in metadata.xml to help euscan do its job more efficiently.

euscan heuristics are described in the "How does-it works ?" section.

Examples

$ euscan amatch

  • dev-ruby/amatch-0.2.7 [gentoo]

Ebuild: /home/euscan/local/usr/portage/dev-ruby/amatch/amatch-0.2.7.ebuild Repository: gentoo Homepage: http://flori.github.com/amatch/ Description: Approximate Matching Extension for Ruby

  • SRC_URI is 'mirror://rubygems/amatch-0.2.7.gem'
  • Using: http://rubygems.org/api/v1/versions/amatch.json

Upstream Version: 0.2.8 http://rubygems.org/gems/amatch-0.2.8.gem

$ euscan rsyslog

  • app-admin/rsyslog-5.8.5 [gentoo]

Ebuild: /home/euscan/local/usr/portage/app-admin/rsyslog/rsyslog-5.8.5.ebuild Repository: gentoo Homepage: http://www.rsyslog.com/ Description: An enhanced multi-threaded syslogd with database support and more.

  • SRC_URI is 'http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.5.tar.gz'
  • Scanning: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
  • Scanning: http://www.rsyslog.com/files/download/rsyslog
  • Generating version from 5.8.5
  • Brute forcing: http://www.rsyslog.com/files/download/rsyslog/rsyslog-${PV}.tar.gz
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.6.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.7.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.8.8.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz ... [ ok ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.10.0.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.11.0.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz ... [ ok ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz ... [ ok ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz ... [ ok ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.12.0.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.4.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.5.tar.gz ... [ !! ]
  • Trying: http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.6.tar.gz ... [ !! ]

Upstream Version: 5.9.1 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.1.tar.gz
Upstream Version: 5.9.0 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.0.tar.gz
Upstream Version: 5.9.3 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.3.tar.gz
Upstream Version: 5.9.2 http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.9.2.tar.gz

Hidden settings

You can configure some settings using the command line, but the init.py file of the euscan package contains more settings, including blacklists and default settings.

Maybe we should add the ability to use /etc/euscan.conf and ~/.config/euscan/euscan.conf to override these settings.

euscan-www: euscan as a service

euscan-www is a web application that aggregates euscan results. For example there is an instance of euscan-www that monitors gentoo-x86 + some official overlays currently hosted at http://euscan.iksaif.net/ .

euscan-www uses django and provides some custom commands to feed the database. You can use euscan-www on you system tree, or preferably you can use a local tree to avoid messing with your system.

Installation

Like any django web app, just start by editing settings.py and then run these two commands.

$ python manage.py syncdb $ python manage.py migrate

Now your instance is ready, you can just run this command to browse it. If you want to host it publicly you should use a real webserver.

$ python manage.py runserver

Creating a local tree

Create a local tree with all that portage (and layman would need). There is an example in euscanwww/scripts/local-tree/. See escan-update.sh to know what env variables you need to run any portage related command in this local tree.

Scanning process

The scanning process is done by euscan-update.sh. You should read carefully this script, and adapt it to your needs. For example it uses gparallel to launch multiple process at a time, and you should adapt that to your number of cpu and network bandwith.

Once your euscan-update.sh is ok, just run it.

$ sh euscan-update.sh

Custom Django management commands

euscan-www povides some new management commands, here is a short description of these commands. Use "help" or read euscan-update.sh to get more informations.

list-packages

List packages stored in database.

scan-portage

Scan the portage tree and store new packages and versions in the database.

scan-metadata.py

Scan metadata and looks for homepage, maintainers and herds.

scan-upstream

Scan upstream package. The prefered way to use this script it to first launch euscan on some packages, store the result of the file, and feed this command with the result.

update-counters

Update statistics and rrd files.

regen-rrds

If you deleted your rrd files, this script will use the database to regen them.

How does it work ?

euscan has different heuristics to scan upstream and provides multiple "handlers". First, here is a description of the generic handler.

Scanning directories

The first thing to do is to scan directories. It's also what uscan do, but it uses a file that describe what url and regexp to use to match packages.

euscan uses SRC_URI and tries to find the current version (or part of this version) in the resolved SRC_URI and generate a regexp from that.

For example for app-accessibility/dash-4.10.1, SRC_URI is: mirror://gnome/sources/dasher/4.10/dasher-4.10.1.tar.bz2 euscan will scan pages based on this template: http://ftp.gnome.org/pub/gnome/sources/dasher/${0}.${1}/dasher-${PV}.tar.bz2

Then, from that, it will scan the top-most directory that doesn't depend on the version, and try to go deeper from here.

Brute force

Like when scanning directories, a template of SRC_URI is built. Then euscan generate next possible version numbers, and tries to download the url generated from the template and the new version number.

For example, running euscan on portage/app-accessibility/festival-freebsoft-utils-0.6: SRC_URI is 'http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.6.tar.gz' Template is http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-${PV}.tar.gz Generate version from 0.6: 0.7, 0.8, 0.10, ... Try new urls: http://www.freebsoft.org/pub/projects/festival-freebsoft-utils/festival-freebsoft-utils-0.7.tar.gz, etc..

Blacklists

euscan uses blacklist for multiple purposes.

BLACKLIST_VERSIONS

For versions that should not be checked at all. sys-libs/libstdc++-v3-3.4 is good example because it's a package which version will always be 3.4 (Compatibility package for running binaries linked against a pre gcc 3.4 libstdc++).

BLACKLIST_PACKAGES

Some packages are dead, but SRC_URI refers to sources that are still being updated, for example: sys-kernel/xbox-sources that uses the same sources as vanilla-sources but is not updated the same way.

SCANDIR_BLACKLIST_URLS

For urls that are not browsable. mirror://gentoo/ is a good example: it's both stupid to scan it and very long/expensive.

BRUTEFORCE_BLACKLIST_PACKAGES and BRUTEFORCE_BLACKLIST_URLS

Disable brute force on those packages and urls. Most of the time it's because upstream is broken and will answer HTTP 200 even if the file doesn't exist.

ROBOTS_TXT_BLACKLIST_DOMAINS

Don't respect robots.txt for these domains (sourcefourge, berlios, github.com).

Site handlers

Pecl/PEAR

A site handler that uses the Pecl/PEAR rest API (http://pear.php.net/manual/en/core.rest.php).

Rubygems

This one uses rubygems's json API (http://guides.rubygems.org/rubygems-org-api/)

Pypy

Uses pypy's XML rpc API.

More ?

An euscan instance is available at http://euscan.iksaif.net/, it scans gentoo-x86 and some overlays every weeks. The source code of euscan is available at http://git.iksaif.net/. 

dimanche, novembre 28 2010

eeepc-wmi's back in the game

Last week, I had an Eeepc 1005PE on my desk and I was finally able to work on eeepc-wmi. I added touchpad led support, fixed conflict with eeepc-laptop and rfkill support for wlan, bluetooth and 3g.

Now, please switch back to eeepc-wmi and remove acpi_osi=(Linux|!Windows 2009):

Why a new eeepc platform driver ?

The eeepc-laptop module use an ACPI device (ASUS010) which have usually the same interface in all eeepc. For example the WLDS will always control the wlan card.

But on newer eeepc, Asus introduced Windows 7 support. Models with windows 7 support have a new WMI interface. And when an OS is detected as Windows 7, the old ASUS010 interface is disabled.

OSI is an ACPI method provided by the operating system that can be invoked by ACPI BIOS code. It is used by BIOS developers to detect which operating system is running. The Linux kernel returns true when OSI is invoked with any known Windows OS string.

So, Linux is identified as Windows 7, ASUS010 device is disable, eeepc-laptop can't be loaded.

Why acpi blacklisting is not a solution ? (or acpi_osi="!Windows 2009")

A quick workaround is to boot with acpi_osi="!Windows 2009" (and sometime acpi_osi="Linux") but:

From Len Brown, maintainer of Linux ACPI subsystem

   Not necessarily the right fix. We have gone to a lot of trouble to discourage BIOS vendors from depending on the ill-defined OSI, so I hesitate to invoke it -- even for a workaround.

Anyway, no that thw wmi interface is here, we can't bet that the old interface won't be maintained and won't expose all features.

What models are affected ?

All Eeepc with windows 7 support (>= 1000).

Status:

Currently, a lot of distributions are using acpi_osi="Linux" with eeepc-wmi blacklisted or worse, loaded *with* eeepc-laptop ! They did that because eeepc-wmi was either unavailable or not finished (this is mainly because acpi4asus project was lacking hardware to develop the driver, let's skip that, but donations are still more than welcome !).

What next ?

In latest kernel, eeepc-wmi should support all important features:

  • backlight
  • hotkeys
  • cpufv (S.H.E.)
  • rfkill (wlan, bluetooth, 3g)
  • touchpad led
  • debugfs (to test not yet supported features without patching the kernel).

So if you're a distrbution developper, please switch back to eeepc-wmi, and remove that ugly acpi_osi=. eeepc-laptop is really not done to work on windows 7 aware Eeepcs.

If you're just a user, and you think eeepc-wmi is lacking something, please open a bug report ( http://dev.iksaif.net/projects/acpi4asus/issues or kernel bugzilla, but CC me ) or use the platform-driver-x86@vger.kernel.org mailing list to send patchs.

lundi, octobre 25 2010

QtTvDB 0.2, Series Watcher 0.1.1, Lugdulo'V 0.3 and QCommandLine 0.2 released !

QtTvDB

Library to use thetvdb.com XML API with Qt. You'll need an API key to use this software, you can get one from thetvdb.com. Used by Series Watcher.

New in 0.2.0:

  • Guest list is now a QStringList

Website: QtTvDB

Series Watcher

Qt clone of SeriesFinal, not only for Maemo but for all platforms supported by Qt. It's a TV series browser and tracker application. Its goal is to help you manage the TV shows you watch regularly and keep track of the episodes you have seen so far. The shows and episodes can be retrieved automatically by using the “TheTVDB API”.

Series Watcher is a free software. It's available on multiple platforms: Windows, Mac-OS X, Linux, Maemo. We're also working on Symbian and iOS ports.

Website: Series Watcher

Lugdulo'V

Find a bike quickly in your city, with map and geolocalisation support ! Currently available for 110 cities in 14 countries, but you're free to write your own plugins, and if your city is not supported, send us a bug report ! To see the list of supported bike sharing network, check our plugins page.

New in 0.3:

  • Symbian port
  • Small bug fixes (Montreal plugin, Plugins Dialog, etc..)
  • Offline mode on Maemo 5

Website: Lugdulo'V

Symbian port

Lugdulo'V source code has been ported to Symbian, but there is no binary available at the moment. It should be available through OVI store very soon.

QCommandLine

Command line parser for Qt (like getopt). Features include options, switchs, params and automatic --version/--help generation.

New in 0.2:

  • Fixed some parsing bugs

Website: QCommandLine

lundi, août 2 2010

QEMU: adaptive Tight, ZRLE and ZYWRLE

Adaptive Tight

I recently added an infrastructure allowing to know the average update frequency for a given region on the screen. Basically, I store last 10 updates times for each 64x64 regions, and only count updates times that happened in the lasts 5 seconds.

Then, I plugged this infrastructure with tight, and adaptive tight is born. The main difference with a normal tight encoding, is that all updates are lossless by default, but if a region is updated a lot, then it'll switch to jpeg encoding.

ZRLE and ZYWRLE

ZRLE and ZYRLE are now supported by QEMU ! ZRLE is a basic run length encoding compressed with zlib, so it's lossless. ZYWRLE is a little more complex, and is well described here: http://micro-vnc.jp/research/remote_desktop_ng/ZYWRLE/publications/ . ZYWRLE is a lossy encoding.

Bandwidth Charts

Here are some results comparing ZRLE, ZYWRLE, and adaptive tight, while playing the big buck bunny trailler.

tight-zrle-zywrle.png

tight5-zywrle5.png

ZYWRLE have very good results, but the main issue is that it's a lossy encoding, and all the screen is lossy if you use zywrle. But that won't be an issue for long with the new adaptive infrastructure :).

Bellow are some screenshots of the different encodings.

Tight

Tight 5, Tight 8, Tight NoJpeg

bunny-tight5.png bunny-tight8.png bunny-tight.png

ZYWRLE

ZYWRLE 2, ZYWRLE 5, ZYWRLE 8

bunny-zywrle2.png bunny-zywrle5.png bunny-zywrle8.png

Static Images

Tight 5, ZYWRLE 5, ZYWRLE 8

kde-tight5.png kde-zywrle5.png kde-zywrle8.png

lundi, juin 14 2010

QEMU: Tight PNG and some profiling

http://github.com/kanaka/noVNC/ developer kanaka recently contacted me. noVNC is a HTML-5 VNC client, that means it's (almost) all in javascript, and only runs in a web browser. He needed an encoding that could be lossless and would allow a fast render in a web browser. We decided to try to add PNG to tight encoding.

Patchs can be found in my git tree, but nothing is standardized, nor merged in upstream QEMU,

Here was the first results with the non-threaded VNC server:

tight-png-sync.png

As we can see, PNG really slows down the guest, and the bandwidth is low only because a lot of frames are skipped and/or delayed.

So, I tried with the threaded VNC server.

tight-png-async.png

And already way better, but a lot of frames were skipped, again.

Then, kanaka tried to disable png fitlers, and here is the result:

tight-png-filters.png

Without filters, the video was a lot smoother, and it's mainly why the bandwidth is higher: more frames were sent !

During these tests, I also wanted to profile QEMU. I tried callgrind, oprofile, gprof, but none of them did what I want (callgrind is actually a great tool, but it was really too slow with QEMU). I finally tried perf (/usr/src/linux/tools/perf/), and it's really great ! No overhead, and really simple to use :

 perf record -f -g -p <pid of vnc thread>
 perf report

jeudi, juin 3 2010

QEMU: Tight JPEG compression

I finished adding Tight JPEG subencoding to QEMU (patchs are comming soon). Here are the first results:

Kubuntu boot

tight-jpeg-boot.png

Big buck bunny 33sec trailer

tight-jpeg-bigbuck.png

I used the non-threaded VNC server for these graph, and I cheated a little with the charts to make them overlap correctly,

I'll try to benchmark the thread and non-threaded server with jpeg as soon as I can get another machine to test that.

vendredi, mai 28 2010

QEMU: Threaded VNC Server results

Another subject of the GSoC was:

VNC server coroutine

Our builtin VNC server is implemented as a combination of a pure state machine to allow scalability. The VNC protocol is very complex though and this implementation is difficult to follow as a result. This project would convert to the existing server to be implemented as a cooperatively scheduled thread in the style of a coroutine. Time permitting, a pure coroutine implementation using ucontext could be implemented. Also, given sufficient progress, one could pursue running the VNC server thread in parallel with the QEMU IO thread in order to achieve better overall scalability.

Last week I decided to try to do that. My implementation use a worker thread and a queue of encoding jobs. The main thread produce encoding jobs and queue them, then the worker thread consume them as soon as possible. Resize (vnc_dpy_resize) and Copy (vnc_dpy_cpy) are still synchroneous at the moment. There is also a lock to block encoding process while updating the display to avoid screen corruptions.

The code can be found on my git tree and will be cleaned and sent soon to qemu-dev mailing list.

Here are some preliminary results. I created a simple image with Suse Studio which just boot and halt.

VNC Sync VS Async

I also made some test while playing the Big Buck Bunny Trailer, the 33sec video takes between 37 and 40 sec to play with vnc ! With the threaded vnc server, it takes between 34 and 35 seconds. But videos are something ticky, and I've got another project for them: adding some kind of mpeg subencoding to VNC :)

mardi, mai 18 2010

QEMU: some charts on tight zlib and hextile

Again, some charts to show some tight results.

I recorded the Kubuntu boot process (showing kpackagekit and a notification at the end). I used tcpdump to record the data and a small python script to convert tcpdump output into .csv files. You can find this small script here: tcpstat.py

zlib-tight-hextile.png

zlib-tight.png

You can see that tight is slightly better than zlib for this workload, but remember that it currently doesn't support gradient filter and jpeg encoding. There is just one thing to remenber: don't use hextile !

mardi, mai 11 2010

GsoC 2010 QEMU: First (dumb) tight benchmarks

For my first (and probably last) summer of code, I've been selected to work on QEMU VNC server encodings. QEMU's built-in VNC server only supports the Hextile encoding. Hextile only uses tile-based compression. Other encodings are more popular such as Tight and ZRLE that use zlib-based compression. These encodings significantly reduce the bandwidth required by VNC.

I started a basic tight implementation (mostly stolen from tightvnc/tigervnc/libvncserver, which all share the same tight.c). But the code have been rewritten to match QEMU coding style, and I'll continue to clean it and improve it.

So, QEMU tight encoding currently support full color rectangles (compressed with zlib), and solid color rectangles (a rectangle with only one color). And here is some preliminary results. I used kubuntu livecd splashscreen, keeping the vnc client open from seconds 29 to 22.

vnc.png

hextile

  • initial rectangle 16384 bytes
  • following rectangles (avg) 139 bytes
  • total 17323 bytes

zlib

  • initial rectangle 13247 bytes
  • following rectangles (avg) 130 bytes
  • total 13247 bytes

tight

  • initial rectangle 10150 bytes
  • following rectangles (avg) 90 bytes
  • total 11345 bytes

Keep in mind that this is not a real benchmark, but tight appears to be the winner for this workload, but zlib is not far and probably eats less cpu.

vendredi, janvier 22 2010

lighttpd configuration: variables and includes

A good way to factorise lighttpd configuration files is to use variables and includes. For example, let's say that you have a lot of vhost, each need specific php configuration.

First, create a _php.conf file:

var.phpconf += " -d session.save_path=" + var.statedir + "/sessions"
 
fastcgi.server = (
        ".php" =>(
                "localhost" =>(
                        "bin-path" => "/usr/bin/php-cgi " + var.phpconf,
                        "socket" => var.rundir + "/php.socket"
                        )
                )
)

Then in your main configuration file:

var.statedir = "/var/lib/lighttpd/"
 
$HTTP["host"] =~ "a.foo.com" {
 var.phpconf = " -d magic_quotes_gpc=Off "
 var.rundir = "/var/run/lighttpd/a.foo.com" 
 include "_php.conf"
}
$HTTP["host"] =~ "b.foo.com" {
 var.phpconf = " -d magic_quotes_gpc=On "
 var.rundir = "/var/run/lighttpd/b.foo.com"
 include "_php.conf"
}

Using that trick, you can do a lot more. For example, here is my configuration file for xf.iksaif.net:

# {{{ variables
var.name     =  "iksaif"
var.basedir  = "/home/iksaif/public_html"
var.logdir   = "/var/log/lighttpd/" + var.name
var.statedir = "/var/lib/lighttpd/" + var.name
var.rundir   = "/var/run/lighttpd/" + var.name
var.username = "iksaif"
var.groupname= "iksaif"
var.docroot  = var.basedir
var.port     = 8081
# }}}
 
include "_base.conf"
include "_php.conf"

iksaif.net migration to lighttp

This week I migrated iksaif.net (and zordania.com, finestown.com, etc...) from apache to lighttpd !

It used to be like described in my previous post: http://xf.iksaif.net/blog/index.php?post/2009/05/08/Securing-a-shared-Web-Server

Now, the architecture is more like that:

lighttpd-misc --- git
                    \- redmine
                    \- lighttpd-iksaif - xf.iksaif.net
                    \- apache          - svn
                                           \- trac
                                           \- finestown
                   \- lighttpd-zordania
                   \- lighttpd-zorddev
                   \- lighttpd-uffs    - blog
                                           \- git
                                           \- redmine

Is not as performant, but it's a lot more flexible. I had to keep apache for some svn repository, because there is no dav-svn module for lighttpd.

I learned a lot, so I'll post some lighttpd configuration examples that could help you one day.

jeudi, octobre 8 2009

Let's Work

Voila maintenant deux semaines que je bosse sur Virtual Browser à commonIT et tout ce passe bien, je suis sur des projs sympa.

A côté de ça, j'ai quand même trouvé un peu de temps pour bosser sur acpi4asus et en particulier pour android-x86. Android-x86, c'est un port d'android pour les "netbooks" et en particulier les EeePC.

J'ai donc backporté eeepc-laptop pour le kernel utilisé par android (un 2.6.29), et là je fais quelques tests avant d'envoyer ça upstream (cwhuang m'ayant gentillement donné un accès en écriture à git.android-x86.org). Pour l'instant, j'ai juste ajouté un patch pour régler la keymap.

Mais bon, le projet en est à ses débuts, alors il y'a vraiment de quoi faire =)

dimanche, septembre 13 2009

Le code et ses raisons : goto en C

Just un post bookmark vers le blog d'rz0 et en particulier un article sur les goto

lundi, août 10 2009

Playing with my EOS400D

Chasse

Chasse

Floc

Floc

Gallery

samedi, juin 13 2009

Poke-a-hole, let's align structs

Just readed . That's a cool tool !

You can find pahole here: http://git.kernel.org/?p=linux/kernel/git/acme/pahole.git :

git clone git://git.kernel.org/pub/scm/linux/kernel/git/acme/pahole.git
cd pahole
mkdir build
cmake ..
make

I tested it on ikmalloc

$ pahole libikmalloc.so
 
struct chunk {
        struct chunk *             main_next;            /*     0     8 */
        struct chunk *             main_prev;            /*     8     8 */
        struct chunk *             free_next;            /*    16     8 */
        struct chunk *             free_prev;            /*    24     8 */
        size_t                     size;                 /*    32     8 */
        size_t                     type;                 /*    40     8 */
        size_t                     asked_size;           /*    48     8 */
        size_t                     alignment;            /*    56     8 */
        /* --- cacheline 1 boundary (64 bytes) --- */
 
        /* size: 64, cachelines: 1, members: 8 */
};

Ok, everything is aligned =).

$ pahole fs/nandfs/nandfs.o --reorganize --show_reorg_steps -C nandfs_io_ops
struct nandfs_io_ops {
        struct nandfs_object *     obj;                  /*     0     8 */
        uint8_t *                  buf;                  /*     8     8 */
        uint32_t                   ofs;                  /*    16     4 */
 
        /* XXX 4 bytes hole, try to pack */
 
        size_t                     len;                  /*    24     8 */
        size_t                     retlen;               /*    32     8 */
        bool                       direct;               /*    40     1 */
 
        /* size: 48, cachelines: 1, members: 6 */
        /* sum members: 37, holes: 1, sum holes: 4 */
        /* padding: 7 */
        /* last cacheline: 48 bytes */
};
 
/* Moving 'direct' from after 'retlen' to after 'ofs' */
struct nandfs_io_ops {
        struct nandfs_object *     obj;                  /*     0     8 */
        uint8_t *                  buf;                  /*     8     8 */
        uint32_t                   ofs;                  /*    16     4 */
        bool                       direct;               /*    20     1 */
 
        /* XXX 3 bytes hole, try to pack */
 
        size_t                     len;                  /*    24     8 */
        size_t                     retlen;               /*    32     8 */
 
        /* size: 40, cachelines: 1, members: 6 */
        /* sum members: 37, holes: 1, sum holes: 3 */
        /* last cacheline: 40 bytes */
}
 
/* Final reorganized struct: */
struct nandfs_io_ops {
        struct nandfs_object *     obj;                  /*     0     8 */
        uint8_t *                  buf;                  /*     8     8 */
        uint32_t                   ofs;                  /*    16     4 */
        bool                       direct;               /*    20     1 */
 
        /* XXX 3 bytes hole, try to pack */
 
        size_t                     len;                  /*    24     8 */
        size_t                     retlen;               /*    32     8 */
 
        /* size: 40, cachelines: 1, members: 6 */
        /* sum members: 37, holes: 1, sum holes: 3 */
        /* last cacheline: 40 bytes */
};   /* saved 8 bytes! */

Wow, cool, 8 byte saved :P

vendredi, mai 8 2009

Securing a shared Web Server

Typical users use shared hosting for their blog, etc ... On this kind of offer, the hoster have multiple clients on the same hardware. As it is impossible to be sure that clients applications are not vulnerable, we should at least be sure that if one is vulnerable, it won't bother the others. We will show you how a bad web server configuration could lead to disaster, and how to secure such a server.

The pdf version can be found at http://xf.iksaif.net/papers/securin....

Lire la suite...

dimanche, mai 3 2009

C Hacks

Some Links:

mercredi, mars 18 2009

Coccinelle: Patchs sémantiques

Je viens de découvrir coccinelle, et c'est vraiment un outil sympa.

Imaginez que vous voulez remplacer tout les expr ? 1 : 0 par des !!expr. Et bah avec coccinelle rien de plus simple :

test.cocci

@@ expression cond; @@
 
- cond ? 1 : 0
+ !!cond
 
@@ expression cond; @@
 
- (!!cond)
+ !!cond

test.c

int main(void)
{
  int val;
 
  val = val ? 1 : 0;
  printf("%d", val);
  printf("%d", val ? 1 : 0);
  printf("%d", (val ? 1 : 0));
}

$ spatch -sp_file test.cocci test.c

--- test.c      2009-03-18 11:06:17.000000000 +0100
+++ /tmp/cocci-output-14620-bafaae-test.c       2009-03-18 11:21:17.000000000 +0100
@@ -2,8 +2,8 @@ int main(void)
 {
   int val;
 
-  val = val ? 1 : 0;
+  val = !!val;
   printf("%d", val);
-  printf("%d", val ? 1 : 0);
-  printf("%d", (val ? 1 : 0));
+  printf("%d", !!val);
+  printf("%d", !!val);
 }

Donc vachement pratique pour changer des trucs quand un search&replace ne suffit pas ! Plus d'informations:

- page 1 de 2