Category Archives: technology

Sparse files are handy sometimes

Disk Space Piechart
I have somewhere an old 2T ext3 volume that holds some disk images, one of which being 1.5T in size (I know, I know). The filesystem was almost full and I needed some extra space to copy off some data. So I need to delete something in order to free up some space, right ? Wrong.

zerofree and fallocate to the rescue! [1]

root@host :/volume/images# zerofree -v 100/vm-100-disk-1.raw
221330954/236263080/402653184
root@host:/volume/images# fallocate -v --dig-holes 100/vm-100-disk-1.raw
100/vm-100-disk-1.raw: 895.5 GiB (961573908480 bytes) converted to sparse holes.

Now I have the free space to use as I wish!

Someday I must convert this volume to LVM-Thin.

[1]
root@host:~# apropos zerofree fallocate
fallocate (1) – preallocate or deallocate space to a file
zerofree (8) – zero free blocks from ext2, ext3 and ext4 file-systems

How to close a JIRA ticket using the REST api

logoJIRAPNGI had to close a bunch of tickets during a cleanup operation and I thought I better spend a few hours learning something new instead of mindlessly clicking buttons into a web interface and waiting for it to refresh. So I set out to learn about JIRA’s REST api. Using python, because why not, I want to learn python as well.

Fortunately Atlassian has pretty good docs and tutorials on it’s webpages. Unfortunately our JIRA instance requires you to fill out the time spent and some custom fields when you want to transition a ticket to the “Resolved” state.

The following hackish script is the result. It will require the username and password for the JIRA instance, the ticket, the time spent and a comment as parameters. Proper argument and error handling is left as an exercise to the reader.

#!/usr/bin/python
import urllib
import urllib2
import base64
import json
import sys

if (len(sys.argv) != 6):
     print "Usage: closeticket.py JiraUser JiraPassword JiraTicket TimeSpent \"Comment text\""
     sys.exit(2)

username = sys.argv[1]
password = sys.argv[2]
key = sys.argv[3]
timespent = sys.argv[4]
comment = sys.argv[5]

# Modify the url to suit your jira instance
url = 'https://jira.url/rest/api/2/issue/%s/transitions' % key
auth = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
data = json.dumps({
  "update": {
    "assignee": [
      {
        "set": {
          "name": username
        }
      }
    ],
    "comment": [
      {
        "add": {
          "body": comment
        }
      }
    ],
    "worklog": [
      {
        "add": {
          "timeSpent": timespent
        }
      }
    ],
    "customfield_10513": [
      {
        "set": [
          {
            "id": "17102"
          }
        ]
      }
    ],
    "customfield_11612": [
      {
        "set": {
          "id": "21903",
          "child": {
            "id": "21904"
          }
        }
      }
    ]
  },
  "transition": {
    "id": "5"
  }
})

request = urllib2.Request(url, data, {
    'Authorization': 'Basic %s' % auth,
    'Content-Type': 'application/json',
})
print urllib2.urlopen(request).read()

Note: custom fields are a bitch. You can use the following to figure them out:

https://jira.url/rest/api/2/issue/MY-ISSUE/transitions?expand=transitions.fields

CentOS woes going from 7 to 7.1 on OpenVZ

centosvz

In case you are running CentOS 7 on OpenVZ and you update to 7.1 you might have a nasty surprise if you reboot the container. The ip adresses configured are no longer applied.

More details at https://bugzilla.redhat.com/show_bug.cgi?id=1207975

TL;DR: you need to apply the following patch to ifup-aliases in /etc/sysconfig/network-scripts:

--- ifup-aliases.orig	2015-04-01 08:46:08.179879018 +0200
+++ ifup-aliases	2015-04-01 08:46:52.558427785 +0200
@@ -261,7 +261,8 @@
                is_available ${parent_device} && \
                ( grep -qswi "up" /sys/class/net/${parent_device}/operstate ||  grep -qswi "1" /sys/class/net/${parent_device}/carrier ) ; then
                    echo $"Determining if ip address ${IPADDR} is already in use for device ${parent_device}..."
-				   if ! /sbin/arping -q -c 2 -w ${ARPING_WAIT:-3} -D -I ${parent_device} ${IPADDR} ; then
+				   /sbin/arping -q -c 2 -w ${ARPING_WAIT:-3} -D -I ${parent_device} ${IPADDR}
+				   if [ $? = 1 ]; then
 					   net_log $"Error, some other host already uses address ${IPADDR}."
 					   return 1
 				   fi

 

CentOS 6 – PHP upgrade – the easy way

phplogoCentOS 6 has PHP 5.3 in the default repository. I’ve got a request from someone to help him upgrade it’s PHP to 5.4 so here is a quick and lazy way to do it.

Start with your typical CentOS 6 with PHP hosting a bunch of sites  all rpms were installed from the default repo:

[root@www ~]# rpm -qa | grep php
php-common-5.3.3-40.el6_6.i686
php-5.3.3-40.el6_6.i686
php-process-5.3.3-40.el6_6.i686
php-cli-5.3.3-40.el6_6.i686
php-mysql-5.3.3-40.el6_6.i686
php-mbstring-5.3.3-40.el6_6.i686
php-gd-5.3.3-40.el6_6.i686
php-enchant-5.3.3-40.el6_6.i686
php-pear-1.9.4-4.el6.noarch
php-pear-Net-Socket-1.0.10-1.el6.noarch
php-pear-Net-SMTP-1.6.1-1.el6.noarch
php-IDNA_Convert-0.8.0-1.el6.noarch
php-pear-Mail-mimeDecode-1.5.5-1.el6.noarch
php-pear-Auth-SASL-1.0.6-1.el6.noarch
php-pdo-5.3.3-40.el6_6.i686
php-xml-5.3.3-40.el6_6.i686
php-PHPMailer-5.2.2-1.el6.noarch
php-pear-Mail-Mime-1.8.4-1.el6.noarch
php-pear-Mail-1.2.0-1.el6.noarch
php-simplepie-1.3.1-3.el6.noarch

I’ve got a tip from a friend (thanks wolfy) about the IUS repo. Bonus points, it’s SafeRepo Aware. So let’s add it to the list of repositories:

[root@www ~]# yum -y install http://dl.iuscommunity.org/pub/ius/stable/CentOS/6/x86_64/ius-release-1.0-13.ius.centos6.noarch.rpm
Loaded plugins: downloadonly, fastestmirror, priorities
Setting up Install Process
ius-release-1.0-13.ius.centos6.noarch.rpm | 7.9 kB 00:00 
Examining /var/tmp/yum-root-5nBZSW/ius-release-1.0-13.ius.centos6.noarch.rpm: ius-release-1.0-13.ius.centos6.noarch
Marking /var/tmp/yum-root-5nBZSW/ius-release-1.0-13.ius.centos6.noarch.rpm to be installed
Loading mirror speeds from cached hostfile
 * base: ftp.ines.lug.ro
 * epel: fedora.mirrors.telekom.ro
 * extras: ftp.ines.lug.ro
 * updates: ftp.ines.lug.ro
Resolving Dependencies
--> Running transaction check
---> Package ius-release.noarch 0:1.0-13.ius.centos6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
 ius-release noarch 1.0-13.ius.centos6 /ius-release-1.0-13.ius.centos6.noarch 8.3 k

Transaction Summary
===================================================================================================================================================================================================================
Install 1 Package(s)

Total size: 8.3 k
Installed size: 8.3 k
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
 Installing : ius-release-1.0-13.ius.centos6.noarch 1/1 
 Verifying : ius-release-1.0-13.ius.centos6.noarch 1/1 

Installed:
 ius-release.noarch 0:1.0-13.ius.centos6 

Complete!

Now we have a bunch of PHP versions we can choose from:

[root@www ~]# yum search php | grep 'cli\.'
php-cli.i686 : Command-line interface for PHP
php54-cli.i686 : Command-line interface for PHP
php55u-cli.i686 : Command-line interface for PHP
php56u-cli.i686 : Command-line interface for PHP

There is only one problem. These packages are conflicting with the existing ones, so just “yum install php54” will not work. So we need one more thing:

[root@www ~]# yum -y install yum-plugin-replace
Loaded plugins: downloadonly, fastestmirror, priorities
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: ftp.ines.lug.ro
 * epel: fedora.mirrors.telekom.ro
 * extras: ftp.ines.lug.ro
 * ius: download.srv.ro
 * updates: ftp.ines.lug.ro
Resolving Dependencies
--> Running transaction check
---> Package yum-plugin-replace.noarch 0:0.2.7-1.ius.centos6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
 yum-plugin-replace noarch 0.2.7-1.ius.centos6 ius 17 k

Transaction Summary
===================================================================================================================================================================================================================
Install 1 Package(s)

Total download size: 17 k
Installed size: 35 k
Downloading Packages:
yum-plugin-replace-0.2.7-1.ius.centos6.noarch.rpm | 17 kB 00:00 
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
 Installing : yum-plugin-replace-0.2.7-1.ius.centos6.noarch 1/1 
 Verifying : yum-plugin-replace-0.2.7-1.ius.centos6.noarch 1/1 

Installed:
 yum-plugin-replace.noarch 0:0.2.7-1.ius.centos6 

Complete!

Now we can just replace the existing packages with:

[root@www ~]# yum -y replace php --replace-with php54
Loaded plugins: downloadonly, fastestmirror, priorities, replace
Replacing packages takes time, please be patient...
Loading mirror speeds from cached hostfile
 * base: ftp.ines.lug.ro
 * epel: fedora.mirrors.telekom.ro
 * extras: ftp.ines.lug.ro
 * ius: download.srv.ro
 * updates: ftp.ines.lug.ro

WARNING: Unable to resolve all providers: ['mysql.so', 'mysqli.so', 'pdo_mysql.so', 'curl.so', 'fileinfo.so', 'json.so', 'phar.so', 'php-core', 'php-ereg', 'php-phar', 'php-standard', 'zip.so', 'pdo.so', 'pdo_sqlite.so', 'sqlite3.so', 'config(php)', 'mbstring.so', 'dom.so', 'wddx.so', 'xmlreader.so', 'xmlwriter.so', 'xsl.so', 'gd.so', 'posix.so', 'sysvmsg.so', 'sysvsem.so', 'sysvshm.so', 'enchant.so']

Resolving Dependencies
--> Running transaction check
---> Package php.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-cli.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-common.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-enchant.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-gd.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-mbstring.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-mysql.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-pdo.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-pear.noarch 1:1.9.4-4.el6 will be erased
---> Package php-process.i686 0:5.3.3-40.el6_6 will be erased
---> Package php-xml.i686 0:5.3.3-40.el6_6 will be erased
---> Package php54.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-cli.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-common.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-enchant.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-gd.i686 0:5.4.40-1.ius.centos6 will be installed
--> Processing Dependency: libt1.so.5 for package: php54-gd-5.4.40-1.ius.centos6.i686
---> Package php54-mbstring.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-mysql.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-pdo.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-pear.noarch 1:1.9.5-2.ius.centos6 will be installed
---> Package php54-process.i686 0:5.4.40-1.ius.centos6 will be installed
---> Package php54-xml.i686 0:5.4.40-1.ius.centos6 will be installed
--> Running transaction check
---> Package t1lib.i686 0:5.1.2-6.el6_2.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================================================================
 Package Arch Version Repository Size
===================================================================================================================================================================================================================
Installing:
 php54 i686 5.4.40-1.ius.centos6 ius 2.7 M
 php54-cli i686 5.4.40-1.ius.centos6 ius 2.6 M
 php54-common i686 5.4.40-1.ius.centos6 ius 916 k
 php54-enchant i686 5.4.40-1.ius.centos6 ius 18 k
 php54-gd i686 5.4.40-1.ius.centos6 ius 116 k
 php54-mbstring i686 5.4.40-1.ius.centos6 ius 906 k
 php54-mysql i686 5.4.40-1.ius.centos6 ius 106 k
 php54-pdo i686 5.4.40-1.ius.centos6 ius 92 k
 php54-pear noarch 1:1.9.5-2.ius.centos6 ius 383 k
 php54-process i686 5.4.40-1.ius.centos6 ius 35 k
 php54-xml i686 5.4.40-1.ius.centos6 ius 142 k
Removing:
 php i686 5.3.3-40.el6_6 @updates 3.3 M
 php-cli i686 5.3.3-40.el6_6 @updates 6.3 M
 php-common i686 5.3.3-40.el6_6 @updates 2.9 M
 php-enchant i686 5.3.3-40.el6_6 @updates 19 k
 php-gd i686 5.3.3-40.el6_6 @updates 312 k
 php-mbstring i686 5.3.3-40.el6_6 @updates 1.9 M
 php-mysql i686 5.3.3-40.el6_6 @updates 189 k
 php-pdo i686 5.3.3-40.el6_6 @updates 150 k
 php-pear noarch 1:1.9.4-4.el6 @base 2.2 M
 php-process i686 5.3.3-40.el6_6 @updates 53 k
 php-xml i686 5.3.3-40.el6_6 @updates 275 k
Installing for dependencies:
 t1lib i686 5.1.2-6.el6_2.1 base 152 k

Transaction Summary
===================================================================================================================================================================================================================
Install 12 Package(s)
Remove 11 Package(s)

Total download size: 8.1 M
Downloading Packages:
(1/12): php54-5.4.40-1.ius.centos6.i686.rpm | 2.7 MB 00:00 
(2/12): php54-cli-5.4.40-1.ius.centos6.i686.rpm | 2.6 MB 00:00 
(3/12): php54-common-5.4.40-1.ius.centos6.i686.rpm | 916 kB 00:00 
(4/12): php54-enchant-5.4.40-1.ius.centos6.i686.rpm | 18 kB 00:00 
(5/12): php54-gd-5.4.40-1.ius.centos6.i686.rpm | 116 kB 00:00 
(6/12): php54-mbstring-5.4.40-1.ius.centos6.i686.rpm | 906 kB 00:00 
(7/12): php54-mysql-5.4.40-1.ius.centos6.i686.rpm | 106 kB 00:00 
(8/12): php54-pdo-5.4.40-1.ius.centos6.i686.rpm | 92 kB 00:00 
(9/12): php54-pear-1.9.5-2.ius.centos6.noarch.rpm | 383 kB 00:00 
(10/12): php54-process-5.4.40-1.ius.centos6.i686.rpm | 35 kB 00:00 
(11/12): php54-xml-5.4.40-1.ius.centos6.i686.rpm | 142 kB 00:00 
(12/12): t1lib-5.1.2-6.el6_2.1.i686.rpm | 152 kB 00:00 
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 8.2 MB/s | 8.1 MB 00:00 
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
 Installing : php54-common-5.4.40-1.ius.centos6.i686 1/23 
warning: /etc/php.ini created as /etc/php.ini.rpmnew
 Installing : php54-cli-5.4.40-1.ius.centos6.i686 2/23 
 Installing : php54-pdo-5.4.40-1.ius.centos6.i686 3/23 
 Installing : t1lib-5.1.2-6.el6_2.1.i686 4/23 
 Installing : php54-gd-5.4.40-1.ius.centos6.i686 5/23 
 Installing : php54-mysql-5.4.40-1.ius.centos6.i686 6/23 
 Installing : 1:php54-pear-1.9.5-2.ius.centos6.noarch 7/23 
 Installing : php54-5.4.40-1.ius.centos6.i686 8/23 
 Installing : php54-mbstring-5.4.40-1.ius.centos6.i686 9/23 
 Installing : php54-xml-5.4.40-1.ius.centos6.i686 10/23 
 Installing : php54-enchant-5.4.40-1.ius.centos6.i686 11/23 
 Installing : php54-process-5.4.40-1.ius.centos6.i686 12/23 
 Erasing : php-5.3.3-40.el6_6.i686 13/23 
 Erasing : php-mysql-5.3.3-40.el6_6.i686 14/23 
 Erasing : php-pdo-5.3.3-40.el6_6.i686 15/23 
 Erasing : php-enchant-5.3.3-40.el6_6.i686 16/23 
 Erasing : php-xml-5.3.3-40.el6_6.i686 17/23 
 Erasing : php-gd-5.3.3-40.el6_6.i686 18/23 
 Erasing : php-process-5.3.3-40.el6_6.i686 19/23 
 Erasing : php-mbstring-5.3.3-40.el6_6.i686 20/23 
 Erasing : 1:php-pear-1.9.4-4.el6.noarch 21/23 
 Erasing : php-cli-5.3.3-40.el6_6.i686 22/23 
 Erasing : php-common-5.3.3-40.el6_6.i686 23/23 
 Verifying : php54-mbstring-5.4.40-1.ius.centos6.i686 1/23 
 Verifying : php54-xml-5.4.40-1.ius.centos6.i686 2/23 
 Verifying : 1:php54-pear-1.9.5-2.ius.centos6.noarch 3/23 
 Verifying : php54-5.4.40-1.ius.centos6.i686 4/23 
 Verifying : php54-gd-5.4.40-1.ius.centos6.i686 5/23 
 Verifying : php54-mysql-5.4.40-1.ius.centos6.i686 6/23 
 Verifying : php54-pdo-5.4.40-1.ius.centos6.i686 7/23 
 Verifying : php54-cli-5.4.40-1.ius.centos6.i686 8/23 
 Verifying : t1lib-5.1.2-6.el6_2.1.i686 9/23 
 Verifying : php54-common-5.4.40-1.ius.centos6.i686 10/23 
 Verifying : php54-enchant-5.4.40-1.ius.centos6.i686 11/23 
 Verifying : php54-process-5.4.40-1.ius.centos6.i686 12/23 
 Verifying : php-common-5.3.3-40.el6_6.i686 13/23 
 Verifying : 1:php-pear-1.9.4-4.el6.noarch 14/23 
 Verifying : php-mysql-5.3.3-40.el6_6.i686 15/23 
 Verifying : php-mbstring-5.3.3-40.el6_6.i686 16/23 
 Verifying : php-process-5.3.3-40.el6_6.i686 17/23 
 Verifying : php-pdo-5.3.3-40.el6_6.i686 18/23 
 Verifying : php-gd-5.3.3-40.el6_6.i686 19/23 
 Verifying : php-5.3.3-40.el6_6.i686 20/23 
 Verifying : php-cli-5.3.3-40.el6_6.i686 21/23 
 Verifying : php-xml-5.3.3-40.el6_6.i686 22/23 
 Verifying : php-enchant-5.3.3-40.el6_6.i686 23/23 

Removed:
 php.i686 0:5.3.3-40.el6_6 php-cli.i686 0:5.3.3-40.el6_6 php-common.i686 0:5.3.3-40.el6_6 php-enchant.i686 0:5.3.3-40.el6_6 php-gd.i686 0:5.3.3-40.el6_6 php-mbstring.i686 0:5.3.3-40.el6_6 
 php-mysql.i686 0:5.3.3-40.el6_6 php-pdo.i686 0:5.3.3-40.el6_6 php-pear.noarch 1:1.9.4-4.el6 php-process.i686 0:5.3.3-40.el6_6 php-xml.i686 0:5.3.3-40.el6_6 

Installed:
 php54.i686 0:5.4.40-1.ius.centos6 php54-cli.i686 0:5.4.40-1.ius.centos6 php54-common.i686 0:5.4.40-1.ius.centos6 php54-enchant.i686 0:5.4.40-1.ius.centos6 php54-gd.i686 0:5.4.40-1.ius.centos6 
 php54-mbstring.i686 0:5.4.40-1.ius.centos6 php54-mysql.i686 0:5.4.40-1.ius.centos6 php54-pdo.i686 0:5.4.40-1.ius.centos6 php54-pear.noarch 1:1.9.5-2.ius.centos6 php54-process.i686 0:5.4.40-1.ius.centos6
 php54-xml.i686 0:5.4.40-1.ius.centos6 

Dependency Installed:
 t1lib.i686 0:5.1.2-6.el6_2.1 

Complete!

The install will not replace your php.ini and will just create an php.ini.rpmnew.

Now we can check if the correct version is installed and restart httpd:

[root@www ~]# php -v
PHP 5.4.40 (cli) (built: Apr 16 2015 18:43:47) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies
[root@www ~]# service httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]
[root@www ~]#

Aaaaand it’s gone done. Enjoy.

If you need so, you can apply the same recipe for php55 or php56.

pfSense on a WatchGuard Firebox

pfsense

 

I was searching for a new gateway box for my home network and a friend gave me an old Watchguard Firebox Edge x1250e to play.

 

20150109_195346

Unfortunately the factory software that came with the box was old and not so feature-rich but reading trough the interwebs I discovered that you can install pfSense on it.

So I preceded in doing so, with bonus geek points that it involved opening the box, updating the BIOS (with some hoops), the works. In the process I upgraded the RAM to 2Gb (from 512Mb) and the CF card to 4Gb (from 512Mb). Here is laying for the operation, with the innards wide open:

20150109_200010

Everything worked perfectly, I even managed to make good use of the LCD output (via the LCDproc package), and now I have a nice firewall at the gate 🙂

I only need to get a hold of a L-shaped power cord because a standard cable is protruding too much out of the back and I have issues mounting it into the rack.

20150109_204640

The last gripe I still have with it is that I need to figure out how to convince the software to forward the igmp messages trough the VPN. But I like a good challenge.

Later edit: It seems that you need to be more specific than “permit ip any any” to convince pf to let the multicast packets trough. It needs “permit ip any any and also packets with ip options set”.

Later edit 2: It seems that “poor man’s error correction” (aka TCP) is actually working. I no longer have stutters in the multicast IPTV streams. (old solution: PIM over GRE, new solution: OpenVPN with TCP transport, this was actually why I was searching for a new box in the first place). Yay!

Tales from the factory – curl vs wget

open_book_02

 

From time to time I will share some stories based on true events, maybe someone will learn something from them. Then again, maybe not. To protect the innocent, some names and events might be edited. Here comes the first one.

 

Someone raised a ticket that their application cannot access a certain url, let’s say “http://My.url.tld”. You dutifully log in to the system in question and try to access the url. Since the app is using the “libcurl” library, you naturally try to test with the respective utility. You confirm that it does not work:

[user@someserver ~]$ curl http://My.url.tld
Error message.
[user@someserver ~]$

In the same time a colleague also sees the ticket but for some reason he does the testing by the way of “wget”. It’s working for him:

[user@someserver ~]$ wget http://My.url.tld
Correct result.
[user@someserver ~]$

You go back and forth with “it’s working”, “no, it’s not” messages until both of you realize that you test differently. So, it’s working with “wget” but not with “curl”. Baffling. What could be wrong ?

After running both utils in debug mode you spot a minute difference:

[user@someserver ~]$ curl -v http://My.url.tld
* About to connect() to My.url.tld port 80 (#0)
* Trying 1.1.1.1... connected
* Connected to My.url.tld (1.1.1.1) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl
> Host: My.url.tld:80
> Accept: */*
>
<
< Error message
<
<
* Connection #0 to host My.url.tld left intact
* Closing connection #0
[user@someserver ~]$

[user@someserver ~]$ wget -d http://My.url.tld
DEBUG output created by Wget 1.12 on linux-gnu.
Resolving My.url.tld... 1.1.1.1
Caching My.url.tld => 1.1.1.1
Connecting to My.url.tld|1.1.1.1|:80... connected.
Created socket 3.
Releasing 0x000000000074fb60 (new refcount 1).

---request begin---
GET / HTTP/1.0
User-Agent: Wget (linux-gnu)
Accept: */*
Host: my.url.tld:80
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK

Correct answer

---response end---
200 OK
Registered socket 3 for persistent reuse.
Length: 242 [text/xml]
Saving to: “filename”

100%[=========================================================================================================================================================================>] 242 --.-K/s in 0s

“filename” saved [242/242]
[user@someserver ~]$

Have you seen it ?

.
.
.
.
.
(suspense drumroll)
.
.
.
.
.
.
.

Turns out that wget is doing the equivalent of an tolower(“url”) so in the actual http request it’s sending “Host: my.url.tld” and curl it’s just taking what I specified in the command line, namely “Host: My.url.tld”. Taking the test test further it turns out that calling curl with the “only lowercase” url is producing the expected results (i.e. working).

I know what you are thinking, it should not matter how you call an hostname. True. Except that in this story there is an load balancer in the way, who tries (and mostly succeeds) to do smart stuff. Well, it turns out that there was an host-based string match in that load balancer that did not quite matched the mixed-case cases.

But a question remains. What is the correct behavior ? The “curl” or the “wget” one ? I lean on the “curl” approach but maybe I am biased. What do you think ?

Magic

Magic Smoke!For some reason some people (usually the clueless, but not always) assume that if you just buy and install some $DEVICE the problem at hand will go away without any other issue.

How ? Magic!

That’s why the big vendors make the big money. People thinking that just throwing money at the problem will magically make the problem go away. It’s not always someone there to point out that this is not the case or if it is his, his or her opinions are ignored.

Let’s be clear on one thing: the only magic present in $DEVICE is the magic smoke. When the magic smoke goes out, the $DEVICE stop working.