Productivity Sync Just another WordPress weblog

April 25, 2014

Hacking coreboot, using dediprog to re-flash C710 and discovering SSD FW updates matter

Filed under: Uncategorized — admin @ 10:19 pm

3 weeks ago I buggered up my C710 after trying to get coreboot with a grub2 payload to boot on my laptop with an older Intel SSD.  I’ve been having issues with the SSD not getting seen by the coreboot payloads without some funny busyness (more on this issue later.)  Anyway after bricking it I put things aside for a few weeks and got back to it last night.  I’m now back to running the prebuilt coreboot from John Lewis that does both GRUB2 and SeaBIOS and finishing up the installation tweaks for ubuntu 14.04 LTS (64 bit)  It was booting a little goofy.

It seems that the older Intel SSD (INTEL SSDSA2M160G2GC (2CV102HD) circa 2009 ) isn’t visible to SeaBIOS unless GRUB2 attempts to probe it and fails.  I suspect there is a quirk I need to do or maybe a SSD FW upgrade I need and that this issue is almost certainly unique to my setup. And, if I swap in a normal HD or a different SSD I’m confident it will just work.  (Hmm, lets try that FW upgrade and see…. sonava@#@%!  after running the Intel® SATA Solid-State Drive Firmware Update Tool  and updating the FW now that strangeness does not happen.  I bet my build of coreboot will work just fine no too!  Lets find out… ok it almost works.  my build boots but losses the mouse pad.  I probably have a config option messed up.)

Ok, well to flash a buggered C710 using a clip and a dediprog wire the clip to the pins on the dediprog the so:

pin 1 === CS
pin 2 === mso
pin 3 === I/O2
pin 4 === gnd
pin 5 === msi
pin 6 === clk
pin 7 === N/C
pin 8 === 3.3v

Follow the instructions from last years OSCON tutorial on coreboot hacking.  (based on the c710)  Every screw except the 2 holding the heat pipe to the main board needed to come out.  (I think it was more like 21 screws)

done.

FWIW I did try to use by buspirate 4.5 to do this as well.  But, it failed the read test.  It was able to see the windbound spi chip but it seemed to fall over / hang when reading the 8MB.  Maybe I need to update its FW too?

http://www.nexflash.com/hq/enu/ProductAndSales/ProductLines/FlashMemory/SerialFlash/W25Q64FV.htm

 

Now I just need to get the brightness controls to work right again (not a coreboot issue.  I re-installed ubuntu 14.04 and need to re-enable the key mappings I had in my 13.10 install I guess I’ll look at the post install instructions now.)

 

February 1, 2014

post 14.04 install configs

Filed under: Uncategorized — admin @ 5:12 pm
Assuming a 64 bit install I like to have the following packages on all my systems

sudo apt-get install vim-gnome tmux cscope ctags vim-doc minicom git-core curl mutt \
  ca-certificates ssmtp msmtp fetchmail procmail ssh sshfs xchat openssl \
  nmap manpages-dev manpages-posix manpages-posix-dev quilt subversion \
  mercurial ipython gcc-avr gcc avr-libc gcc-avr binutils-avr avr-libc \
  avrdude python-numpy python-matplotlib graphviz python-scipy \
  python-numpy-doc python-pip python-virtualenv laptop-mode-tools powertop \
  latencytop libncursesw5-dev gitk rar gimp synaptic libav-tools openjdk-7-jdk
sudo apt-get install git gnupg flex bison gperf build-essential \
  zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \
  libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \
  libgl1-mesa-dev g++-multilib mingw32 tofrodos \
  python-markdown libxml2-utils xsltproc zlib1g-dev:i386


sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1/usr/lib/i386-linux-gnu/libGL.so

sudo apt-get build-dep linux-image-3.13.0-24-generic linux-image-3.13.0-24-lowlatency

sudo apt-get install linux-image-3.13.0-24-lowlatency
sudo dpkg-reconfigure dash (answer no) 

Now the system is acceptable for use and ready for configuring assorted dot-files the way I like them. 

sudo apt-get install unity-tweak-tool ubuntu-restricted-extras indicator-multiload 

sudo sh -c 'echo "deb http://archive.canonical.com/ quantal partner" >> /etc/apt/sources.list' 
sudo apt-get update sudo apt-get install skype 
sudo apt-get install gstreamer0.10-plugins-ugly libxine1-ffmpeg gxine mencoder libdvdread4 \
totem-mozilla icedax tagtool easytag id3tool lame nautilus-script-audio-convert libmad0 mpg321 \
libavcodec-extra 

If you are rocking a C710 running Seabois then you'll want to add the following to your /etc/modules file:
i2c_i801
i2c_dev
chromeos_laptop
cyapa

then install google chrome form their site.

For skype correct behaviour with the notifier icon don't forget to:
sudo apt-get install sni-qt:i386

	

January 26, 2014

Crazy (or Cool?) Python idiom using __getattr__ hackery

Filed under: Uncategorized — admin @ 11:36 am

I was exploring the python-git package and how the version differences between the version of python-git you get using apt-get install python-git on Ubuntu 12.04 and 13.10 seems to break a program I use.  In my investigation I was trying to sort out how the library worked.  I found a new python idiom that I’m having a hard time accepting as a “good thing” or a path to chaos.

The Specifics:

  • ubuntu 12.04 will put version 0.1.6 of the library in your python lib.  The project is hosted at:”git@gitorious.org:git-python/git-python.git”  With version “9a4b1d4 Bumped version 0.1.6”
  • ubuntu 13.04 will put version 0.3.2-rc1 from github remote : “https://github.com/gitpython-developers/GitPython.git” With version “0e9eef4 Bumped version to 0.3.2 RC1”
  • It looks like the development moved to gitbub after 96c7ac239a2c9e303233e58daee02101cc4ebf3d — 0.3.1-beta2.

The API change that broke a program I was using between 0.1.6 and 0.3.2-rc1 was “Repo.commits_between” it was removed.  After some tracing through the 0.1.6 version of the code I find commits_between calls a function “commit.find_all” and then looking at its implementation I see it call repo.git.rev_list(…) function.  And through some grepping I see that there is in fact NO DEFINITION of rev_list in the python code!   This was a head scratcher.

After some more digging I see the repo.git is an object defined by cmd.py of class Git.  and it has a _call_process member that gets invoked as a side effect of __getattr__ and uses a “dashify” trick to change any attempts to call a git command using the convention of using underscores instead of dashes to map to calling the popen-ing the correct program.

Basically with this trick *every* git API is available as a python call.  So repo.git.rev_list maps to a popen call too “git-rev-list”

WRT the loss of “commits_between” function in the library, as this function was a one-liner calling another one-liner it can be replaced with a one-liner  “reverse( Repo.git.rev_list(sha1,sha2))”  FWIW I do agree with the removal of the trivial one liner abstractions in the older git-python version.  Still, I feel abused by the lack of introspection and confusion this implementation has provided.

I don’t know if I should be irritated by this idiom of creating an indeterminate number of callables implicitly or be impressed by the crazy shit I could do with it.

It seems that __getattr__ is the last in a series of python look ups when the interpreter evaluated a statement.  So if there is no existing match it eventually falls through to __getattr__ and at this point you can do almost anything.

Notes:

  • commits_between was replaced by iter_commits on version 0.2 (says the project changes.rst file)
  • The tutorial.rst calls out how to use git directly and calls out what I just learned in my reverse engineering.  Guess I could have read the tutorial first but, what’s the fun in that?  I wouldn’t have learned how it works to provide the direct access to git commands if I stopped there.

 

September 15, 2013

Debug tricks for linux distrobutions

Filed under: Uncategorized — admin @ 9:51 am

This is a quick post to capture some earnings on tips for tracing and debugging behaviors in Ubuntu linux on (ChrUbuntu specifically) and how the issues got root caused and the work arounds.

 

The URL’s of were I picked these are are:

The problem I was having was that my C7 Chromebook running ChrUbuntu was eating its battery after the first resume.  Using TOP it was clear that the process sucking the battery was polkitd.  However; the root case of polkitd going ape shit turns out to be some ACPI interrupts that don’t get shut off after the resume.  These are notes on how to figure things out next time on my own.

  1. First use top to find out what process is burning the CPU.
  2. use sudo strace -p <pid> to see what that process is doing. (NOTE: this doesn’t help in this case but is still interesting.)
  3. use dbus-monitor –system to see what sorts of things are going on from DBus. (NOTE: this doesn’t help much either but still interesting)
  4. disable polkitd and look for the next largest CPU user (chmod 0 <path-to-polkitd>; sudo killall polkitd) .  Ah!  looks like ACPID is using a modest amount of CPU.  Hmm. lets STACE it!.  Hmmm, looks like its getting lid/button events non-stop.
  5. re-enabled polkitd (you need it for a lot of ubuntu desktop to work)
  6. Check the /proc/interrupts for higher than wanted interrupt rates.
  7. Check /sys/firmware/acpi/interrupts/ for high interrupt rates. (grep -r . /sys/firmware/acpi/interrupts/* )  Ah! looks ike gpe1F is going nuts!  lets shut it up.
  8. echo “disable”>/sys/firmware/acpi/interrupts/gpe1F

ok  That helped.  Lets make it stick. http://askubuntu.com/questions/176565/why-does-kworker-cpu-usage-get-so-high has the magic beans.  I used.  Just swap gpe13 for gpe1F and things are happy.

The only down side is that now to suspend the device I need to manually execute: “sudo pm-suspend”  Which is ok IMO as this is a cheep hacking computer for me so it doesn’t need to be perfect.

My next thing to figure out is how to update the chrubuntu kernel to the latest and what interesting tidbits of fun I’ll discover in that.

 

May 5, 2013

Random things this weekend…

Filed under: Uncategorized — admin @ 3:18 pm

Find tricks:

find /sys/devices/ -name driver -exec file “{}” \; | sed “s|^.*\.\.||” | sort -u | wc

 

Building mobi’s from XML files:

I was trying to convert a docbook based document from xml to mobi. http://files.kroah.com/lkn/lkn_xml.tar.gz  to mobi.

The steps are:run dbtoepub after getting the xml / docbook stuff, and any dtd files (say the safary_future.dtd files needed) Then run the kindlegen program http://www.amazon.com/gp/feature.html?ie=UTF8&docId=1000234621 on the ebub file.

FWIW I ended up getting the dtd file by cloning a repo on git hub git://github.com/sebastianbergmann/phpunit-documentation.git and copying its dblite directory to my /opt/tools/docbook/ directory.

Then I still had problems with a few missing xml files so I deleted their reference (metadata.xml, cyprt.xml)  from the book.xml The resulting mobi file looks to work fine on my tablet running the kindle reader apk.

 

Linux kernel spelunking of the /sys directory

The sysf directory has a lot of interesting data in it.  I’m looking to identify all the drivers and modules and devices my system thinks it has in it.  To this end I’ve been using grep and find and now setting up to implement a python program to extract the “drivers” the running kernel thinks it has access too as well as the “device”s  it knows about, and the set of “modules” it has (even if they are not loaded).  I should be able to map the drivers in use to devices and kernel config settings.  I should also be able to identify devices that are missing drivers.  I should be able to identify drivers that really are unneeded.

At least thats what I “feel” I should be able to do.  The reality is there are things happening to complicate my simple assumptions.  As I get into this exploration I’ll fill in the details.  But, by running a few find commands I can identify all the drivers the system thinks it has access too at runtime (by searching for bind in the /sysfs tree using find).  I can also know all the modules the system knows about by listing the /sys/module/ directory.  I can identify all the drivers associated with devices by searching for “driver” using find.  I can find all the devices and drivers that have issues a udev envent by finding all instances of  “modalias” using find.

The problems I have is that the numbers don’t add up yet:

Linux mgross-MOBL 3.9.0 #8 SMP Sun May 5 12:30:05 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux
mgross@mgross-MOBL:~$ find /sys/ -name device -exec file "{}" \; | sed "s|^.*\.\.||" | sort -u | wc
 124 172 2795
mgross@mgross-MOBL:~$ find /sys/ -name driver -exec file "{}" \; | sed "s|^.*\.\.||" | sort -u | wc
 42 44 1121
mgross@mgross-MOBL:~$ find /sys/ -name unbind | wc
 60 63 2226
mgross@mgross-MOBL:~$ find /sys/ -name bind | wc
 62 65 2192

mgross@mgross-MOBL:~$ find /sys/ -name refcnt -exec cat "{}" \; | wc

 101 101 225
mgross@mgross-MOBL:~$ find /sys/ -name refcnt -exec cat "{}" \; | grep -v 0 | wc
 67 67 138
mgross@mgross-MOBL:~$ lsmod | wc
 102 361 4543
mgross@mgross-MOBL:~$ find /sys/ -name modalias | wc
 135 135 8093
mgross@mgross-MOBL:~$ find /sys/ -name module | wc
 33 33 1198
mgross@mgross-MOBL:~$ ls /sys/module/ | wc
 144 144 1332
 

So I have 101 modules loaded and 43 drivers (modules tend to be drivers and I feel the 43 “drivers” should include drivers that are loaded a modules. Yet I can bind/unbind only 62(or 60) drivers from devices and the system thinks there are 144 modules going on.

Something isn’t adding up for me just yet.  Clearly my definition of device, driver, and module I am assuming does not match up with reality here.

 

April 14, 2013

Setting up mod_wsgi to host flaskr

Filed under: Uncategorized — admin @ 5:47 pm

Problems and gotchas:

 

I spent a bunch of time today getting apache2 with mod_wsgi to share up the Flask sample application: “flaskr”  this is posting to capture the issues and solutions so when I look at this again I won’t be starting from scratch.

Setting up the virtual env within the *.wsgi file failed. I tried the activate_this.py thing but it never worked.

I kept getting a syntax error form mod_wsgi in the /var/FLASK/flaskr/flaskr.wsgi file on the line ” import sys”!!!! I eventually worked around it by setting the WSGIPythonPath in the apache2.conf file. (no clue on why it failed or why changing the apache2.conf fixed it)

I forgot to turn my flask app into a “module” the from flaskr import app line of the flaskr.wsgi file was failing because my app was a program an not a module that could be imported. Then I needed to fix up the import path to work with the sys.path I set up.

Finally the search path to the sqlite3 data base was path relative in my flask.py. It needs to be absolute for mod_wsgi to work ok.

yup it was a pain in the butt.

Ref URLs

http://www.subdimension.co.uk/2012/04/24/Deploying_Flask_to_Apache.html
https://code.google.com/p/modwsgi/wiki/VirtualEnvironments
http://flask.pocoo.org/docs/deploying/mod_wsgi/
http://thegnar.org/sync/?p=1
http://thegnar.org/sync/?p=299

Configurations:

wsgi user account:

useradd -M flask
usermod -s /bin/false flask
usermod -L flask
adduser flask www-dat

/etc/apache2/apache2.config:

add the following to the end of the file:

WSGIPythonHome /var/FLASK/venv

/etc/apache2/sites-available/default:

put the following in the <VirtualHost *:80> section

 WSGIDaemonProcess flaskr user=flask group=www-data threads=5 home=/var/FLASK
 WSGIScriptAlias /flaskr /var/FLASK/flaskr/flaskr.wsgi
 <Directory "/var/FLASK/flaskr">
   WSGIProcessGroup flaskr
   WSGIApplicationGroup %{GLOBAL}
   WSGIScriptReloading On
   Order deny,allow
   Allow from all
 </Directory>

Set up working directory and virtualenv to use (do as root):

mkdir /var/FLASK/
 cd /var/FLASK
 virtualenv --no-site-packages venv
 . venv/bin/activate
 pip install Flask
 cp -r /home/<username>/dev/flaskr .
 chown -R flask:flask /var/FLASK

/var/FLASK/flaskr/flaskr.wsgi:

import sys
sys.path.insert(0, '/var/FLASK')
from flaskr.flaskr import app as application

Virtual Box bridge interface and ProxyChains

Filed under: Uncategorized — admin @ 5:27 pm

How to experiment with apache2-mod_wsgi using virtual box:

I wanted to figure out how to deploy Flask applications on my work computer without screwing it up too much with experimentation and hacking around in crazy ass ways.  So I decided to deploy it on a Virtual box ubuntu guest OS.  But to make it work I need the VBox guest to open up port 80 that my host could somehow send http requets too.

This post is basically a SOCKS enhanced version of the bridging information I posted about in 2009 http://thegnar.org/sync/?p=1

The steps are:

  • HOST:
    • sudo apt-get install bridge-utils uml-utilities ssh
    • add to /etc/ssh/sshd_config: AllowTcpForwarding yes
    • create a vboxbridge.sh scrip:
# create a tap
sudo tunctl -t tap1 -u <username>
sudo ip link set up dev tap1
# create the bridge
sudo brctl addbr br0
sudo brctl addif br0 tap1
# set the IP address and routing
sudo ip link set up dev br0
sudo ip addr add 10.1.1.1/24 dev br0
sudo ip route add 10.1.1.0/24 dev br0
    • Install VirtualBox
    • build a ubuntu guest VM from an ISO file.
    • After settup and installations are finished you will Configure it to use bridged networking on the tap1 interface
    • For initial setup you will use NAT to because of proxy issues with bridging to tap1.
  • Guest
    • apt-get install tmux apache2 vim  ssh pyflakes pep8 python-setuptools python-virtualenv proxychains libapache2-mod-wsgi
    • /etc/proxychains.conf: [ProxyList] socks5 127.0.0.1 9999
    • Firefox: about:config (in addressbar):  network.prox.sock_remote_dns true
    • Firefox manual proxy setting:  localhost Port 9999, SOCKSv5
    • ssh -C <host_username>@10.1.1.1 -D 9999 -N

 

with this configuration / set up the Vbox guest is now able to host a web server to 10.1.1.2:80 to the host OS.  Also, the guest web browser works ok and I can use “proxychains apt-get” and proxify other non proxy aware applications using proxychains.

 

 

March 22, 2013

tmux is the bomb

Filed under: Uncategorized — admin @ 2:35 pm

I’ve been a screen user for a few years and just stumbled across tmux and I’m switching.

tmux handles paned windows and resizing so much better and its currently under active development where screen has been stagnant since 2005 (I think).

Getting going with tmux I only really need a handful of commands and for me it seems less cryptic than screen.  BTW I’m sticking with the ctrl-b as the command prefix key because I use minicom and with tmux I can have a minicom session in a screen pane.  yay!

TMUX commands I use:

  • ^b %  — vertical split
  • ^b ”  — horizontal split
  • ^b <arrow key> — move between panes.
  • ^b <alt>+<arrow>  — adjust size of current pane
  • ^b <pg-up>  — scroll up the history buffer
    • when in copy mode ^s for search down (will wrap in buffer)
    • when in search mode n will go to next and N will search the opposite direction
    • q exist page up / copy mode
  • ^b ,   — rename current window tab
  • ^b n — cycle to next window tab
  • ^b p — cycle to previous window tab
  • ^b d  — detach from session
  • tmux att -t <session name>  — attach to session by name
  • tmux ls  — list sessions
  • ^b :break-pane  — move current pane to a tab’ed window
  • ^b :join-pane -t dest-wind  — move current window to dest-wind as a pane.
  • ^b ? brings up a help page of key bindings

 

There are copy and past things but I will use my host to do those outside of tmux so I have not bothered figuring that out.

 

.tmux.conf file:

set -g history-limit 50000                                                                                                                             │
set -g default-terminal screen-256color                                                                                                                │
setw -g aggressive-resize on

 

February 23, 2013

Memory and Power tuning of my HP Envy-17 laptop running ubutnu 12.04

Filed under: Uncategorized — admin @ 3:56 pm

About the box:  home is on an SSD and so is the root.  There is a spinning drive as well.  It has 16GB of 1600MHz RAM.

 

  • http://youtu.be/XloTXFgNL7s
  • installed laptop-mode-tools
  • changed my sysctl.conf to set “vm.swappieness=2”
  • set my fstab to use ramdisks for temp directories
    • tmpfs /tmp       tmpfs noatime,mode=1777,size=1350m 0 0
      tmpfs /var/tmp   tmpfs noatime,mode=1777,size=1350m 0 0
      tmpfs /var/spool tmpfs noatime,mode=1777,size=1350m 0 0
      tmpfs /var/log   tmpfs noatime,mode=0755,size=1350m 0 0
  • mounted my root using “noatime,nodiratime,discard,errors=remount-ro”
  • turned off the discrete graphics in my rc.local !  This dropped my burn rate from 45W to 13W!!
    • echo ‘OFF’ > /sys/kernel/debug/vgaswitcheroo/switch
  •  lastly I put the following into my rc.local to avoid some latencies in the file system.  It may be overkill:
  • for dev in sda sdb sdc
    do
    QSZ=1024
      DEV=/sys/block/$dev/queue/nr_requests
      if [ -r $DEV ]; then
        echo $QSZ > $DEV
      fi
    
    QSZ=1024
      DEV=/sys/block/$dev/queue/read_ahead_kb
      if [ -r $DEV ]; then
        echo $QSZ > $DEV
      fi
    done
    
    

man dpkg example worth remembering

Filed under: Uncategorized — admin @ 7:51 am

copy and paste from the man page for dpkg(1):

To make a local copy of the package selection states:

dpkg --get-selections >myselections

You might transfer this file to another computer, and install it there with:

dpkg --clear-selections
dpkg --set-selections <myselections

Note that this will not actually install or remove anything, but just set the selection state on the requested packages. You will need some other application to actually
download and install the requested packages. For example, run

apt-get dselect-upgrade.
« Newer PostsOlder Posts »

Powered by WordPress