Category Archives: Linux

Accessing SOLR server instance with SSH Tunnelling

Linux

I recently discovered the power of SSH tunneling: a Swiss Army knife for your remote services control when the only access available for your server is SSH.

If your remote server only provides SSH access and you need to access (debug, test, check..) the available services on that machine SSH Tunneling is your rescue. Keeping things simple: SSH let you open a listening socket on your machine, while redirecting all its traffic to a specific port on your SSH-accessed host.

In the following example I will describe this scenario: on my remote server (IP: x.x.x.x) a Solr instance has been configured as a search service, it has been locked to accept only local connections coming from 127.0.0.1, and I need to check its configuration by the Solr admin panel (usually reachable on port 8983). The remote server is only accessible by SSH (port 20) and HTTPS (port 443).

With SSH Tunneling I was able to access the Solr admin panel from my browser: all the data is piped through the web and to the server by SSH.

The command I used is quite simple:

ssh -L localhost:8080:127.0.0.1:8983 user@x.x.x.x -N -C

Where:

  • 8080 is the local port where the tunneling starts from (on machine)
  • 127.0.0.1:8983 is the destination IP:port for my request; in other words where the tunnel points to ( as seen as if the connection starts once logged into the remote server)
  • user@x.x.x.x the remote SSH credentials
  • -N don’t send any command through SSH, simply wait on the shell
  • -C enable SSH protocol compression (we care about speed, don’t we?)

Once the connection has been established and the SSH tunnel build, we can visit “http://localhost:8080” to actually access the remote Solr admin interface.

For a more detailed description of SSH Tunneling use cases, please check SSH manual

Easychair data extraction

Linux,

Have you ever heard of EasyChair? It is a free, simple and efficient way for managing a (scientific) conference: provides you all most of the tools for handling paper submitting, approving and camera-ready submitting (for further details please refer to EasyChair website).

The first issue is that some advanced features (as the complete data access as XML) is only available as a paid service. What if the data you need is already available but only as an (hugly) HTML file? I needed the whole list of accepted papers and the only option was an HTML page, formatted by DIVs and not, as some accessibility rules suggests, as a Table. First solution: copy-n-paste from HTML to a spreadsheet. More advanced: provide a script for converting such file to a “well written” HTML. In the generated file the list of papers are in a HTML table, no stylesheets are applied and all the links to authors webpages are removed.

Here we go: a simple set of sed rules to convert the list of accepted papers to a table based page.
Put all of these in a .sed file and invoke the sed commad as:

#sed -f file.sed < accepted-papers.html > accepted-papers-converted.html

The file.sed contents:

s/<br\/>/ /g
s/<style>.*<\/style>//g
s/<\/h1>/<\/h1><table>/g
s/<\/body>/<\/table><\/body>/g
s/<b>Abstract: <\/b>//g
s/<\/div><div class="paper">/<\/tr><tr><td>/g
s/<div class="paper">/<tr><td>/g
s/<span class="authors"><span>//g
s/<\/span>\. <\/span>/<\/td>/g
s/<span class="authors">//g
s/\. <\/span>/<\/td>/g
s/<span class="title">/<td>/g
s/<\/div><div class="abstract">/<\/td><td>/g
s/<a href="[^"]*">\([^<]*\)<\/a>/\1/gg

MySQL recovery using ibdata and ib_logfile1.. files

Linux, Windows, , ,

I recently had my server out of order and I could only access to files (thanks to providential Linux-on-usb). I manage to backup my MySQL files (ibdata, ib_logfile1, ib_logfile2, and the tables *.frm files). No sql dump to be imported into a new MySQL installation.

I remembered a good tutorial on recovering database structure and data using my backups, but I couldn’t find anymore on the web.. I went step-by-step to recover my data: simply replacing the “data” directory inside the new installation setup will give you errors about InnoDB “sequence numbers” (and MySQL will suggest you to refer to “InnoDB force recovery feature“). Continue reading MySQL recovery using ibdata and ib_logfile1.. files →

Bootable Windows7 usb-key with Linux

Linux, Windows, ,

I recently downloaded Windows7 Professional using the Academy software distribution service (it’s a collaboration between Microsoft and my university). I worked under Linux and I couldn’t make a boot-able DVD or USB-Key using the Windows 7 USB/DVD ISO tool (which I recently used and find out that it’s not working with usb-keys!!)

Even if reading some posts I couldn’t manage to get a working DVD/USB installation media; i followed the steps found here:

I finally get a USB media with Windows7 booting, without any “Missing operating system” or “missing NTLDR” error message. Here my steps  under Ubuntu 10.04 (nb: my USB key is /dev/SDB device) Continue reading Bootable Windows7 usb-key with Linux →

Ubuntu 10.4 and Mercurial-Server (Apache2, mod_wsgi)

Linux,

You know, when you start coding you always (don’t you?) end up looking for a safe-and-easy way to track code changes, code commit log and manage the “back-to-the-past” feature (for example when you start coding a new feature and end up in a completely not working app) 🙂

Sometimes I used free services for repositories (like GitHub, BitBucket, SourceForge) to store my open-source projects (BitBucket plans allow you also to setup a single private repository with 1Gb of free space).

Now I’m setting up something like that for my Lab, allowing students to work from home and share their code with co-workers and professors. I’m happy with Mercurial, so I’ve installed on our web-server (latest Ubuntu Server) all the stuff for serving mercurial repositories with SSH authentication.

Continue reading Ubuntu 10.4 and Mercurial-Server (Apache2, mod_wsgi) →