C++Guns – RoboBlog blogging the bot

27.04.2015

rsync cron ssh - daily compressed remote backup

Filed under: Allgemein — Tags: — Thomas @ 12:04

Add user
Add user on you lokal and remote host. Disable password auth on your remote host:
# adduser --disabled-password user

Create authentication keys on your lokal machine
$ ssh-keygen -t rsa

Copy user/.ssh/id_rsa.pub to the end of remote host user/.ssh/authorized_keys

rsync command
-h humand readable
-P show progress
-a archive mode
-v verbode
-z compress
-e using ssh

sync --stats -hPavze ssh /home/user user@remotehost:/home/user/daily

lokal cron command
Put this in an extra file in /etc/cron.d/
Calls rsync with username user daily at 10:01pm
The current date, standard and error output from rsync is redirected to cronlog file.

1 22 * * * user date >> /home/buchhaltung/cronlog; rsync --delete --stats -hPavze ssh /home/user user@remotehost:/home/user/daily >> /home/user/cronlog 2>&1

remote cron command
Put this in an extra file in /etc/cron.d/
Make a compressed backup every day at 11:01pm
1 23 * * * user tar -cvjf /home/user/dailybackup/daily_$(date +%Y%m%d).tar.bz2 /home/user/daily/

usefull links
http://www.marksanborn.net/howto/use-rsync-for-daily-weekly-and-full-monthly-backups/
http://www.marksanborn.net/linux/learning-cron-by-example/
http://www.linuxproblem.org/art_9.html

24.04.2015

return void func call (Pseudo Bug of the day 3)

Filed under: Allgemein — Tags: — Thomas @ 08:04

Heute gefunden. Im Prinzip geht der pseudobug so:

void calc(int a, int b) {
  std::cout << "func " << a + b << "\n";
}

void add(int a, int b) {
  return calc(a, b);
}

int main() {
  add(2, 3);
  return 0;
}

Na, wer sieht das Problem? Wird calc aufgerufen?

(Ja, aber explicit void zurückgeben is braindamage ;) )

21.04.2015

Passing random generators around (functor)

Filed under: Allgemein — Tags: — Thomas @ 02:04

Update: Version with template instead of std::function C++ Guns: Passing function objects around (Update Example random generators)

Consider the following problem: One has a parallelly callable function which needs a random generator for every thread. One wants to call the function several times and every time we want another random sequence. How can we implement this?

Between every function call the random generator must keep its states. So the caller of the function must own the generator objects and pass them to the function. The caller is also in responsible the initialize the random generators. One can feed them with random seeds. But i think there exist a better solutions [1]

We create an array of n random generators, initialize them with a random seed and pass them per pointer to the function.

Inside this function we need a distribution. A simple rand() % maxrand is not good. It uses only the lower bits which may not be random. With c++11 we can bind the random generator and number distribution together. This allows us the create a nice dice() object which represent our rand(). Of course, this object can be simply pass around.

Output:

Thread 0 sum 5021
Thread 0 sum 4966
Thread 3 sum 4996
Thread 3 sum 5014
Thread 1 sum 5007
Thread 2 sum 4970
Thread 1 sum 5092
Thread 2 sum 5002
Info sizeof std::function 16

Code:


// can be passed by value
int function2(std::function dice) {
  int sum = 0;
  for(int i=0; i < 1000; i++) {
    sum += dice();
  }
  return sum;
}

// pass by reference. it has state
int function(std::minstd_rand& generator) {
  // we need random numbers from 0 to 10
  std::uniform_int_distribution distribution(0,10);

  // we bind the generator and distribution together to a new functor dice()
  // std::bind copy its arguments by value. To change the given generator state,
  // my must pass it by reference. This can be easly done with std:ref.
  // Yeah C++ can be strange.
  std::function dice = std::bind ( distribution, std::ref(generator) );

  return function2(dice);
}

int main() {
  omp_set_num_threads(4);

  // create 4 random generatores with random seed
  std::minstd_rand seedGen;
  std::minstd_rand generators[omp_get_num_threads()];
  for(int i=0; i < omp_get_num_threads(); i++) {
    generators[i].seed(seedGen());
  }

#pragma omp parallel for
  for(int i=0; i < 8; i++) {
    const int id = omp_get_thread_num();
    // pass one generator to our function
    int sum = function(generators[id]);

    std::cout << "Thread " << id << " sum " << sum << "\n";
  }

  std::cout << "Info sizeof std::function " << sizeof(std::function) << "\n";

  return 0;
}

[1] Tina’s Random Number Generator Library
Tina’s Random Number Generator Library

Calling multiple random generator in parallel c++11

Filed under: Allgemein — Tags: — Thomas @ 00:04

Standard random functions are not thread safe. Calling them in parallel may result in undefined behavior or the threads blocking each other. This is a little test to create the same random sequences in serial as in parallel too.

Setting the same seed on a random generator as a start point should produce the same random sequence. First we create one generator and print the first three numbers. Restart it with different seed numbers.
Then we create three random generators with the same seeds and call them parallel. They should create the same sequences even if they are called interleaved.

Program output:

seriell
seed 1
48271
182605794
1291394886

seed 2
96542
365211588
435306125

seed 3
144813
547817382
1726701011

parallel
Thread 0 seed 1: 48271
Thread 0 seed 1: 182605794
Thread 0 seed 1: 1291394886
Thread 1 seed 2: 96542
Thread 1 seed 2: 365211588
Thread 1 seed 2: 435306125
Thread 2 seed 3: 144813
Thread 2 seed 3: 547817382
Thread 2 seed 3: 1726701011

Yep, work as expected :D
The program looks not as simple as it might can be. But I think it is okay.

void waitlong() {
  long a = 0;
  for(int i=0; i < 1000000; i++) {
      a += i;
  }
}

int main() {
  // create some random number and store them in expectedResult
  int expectedResult[3][3] = { 0 };
  std::minstd_rand generator;

  std::cout << "seriell\n";

  for(int id=0; id < 3; id++) {
    generator.seed(id+1);
    std::cout << "seed " << id+1 << "\n";
    for(int i=0; i < 3; i++) {
      expectedResult[id][i] = generator();
      std::cout << expectedResult[id][i] << "\n";
    }
  }

  // create three generators with the same seed and call them parallel
  std::cout << "\nparallel\n";

  std::minstd_rand generators[3];
  generators[0].seed(1);
  generators[1].seed(2);
  generators[2].seed(3);

  int result[3][3] = { 0 };
  int count = 0;

  // create 3 threads and interleave them with schedule(static, 1) num_threads(3)
  // We can not call cout in a threaded enviroment. So we must store the random numbers
  // in a array and print them laster. count and result is for this.
#pragma omp parallel for firstprivate(count) shared(result) schedule(static, 1) num_threads(3)
  for(int i=0; i < 3*3; i++) {
    int id = omp_get_thread_num();
    int rand = generators[id]();

    // not error free even with omp critical
//     std::cout << "Thread " << id << " seed " << id+1 << ": " << rand << "\n";

    result[id][count++] = rand;

    // wait a litte bit to give the next thread a chance to start
    waitlong();
  }

  // now print and compre the results
  for(int id=0; id < 3; id++) {
    for(int i=0; i < 3; i++) {
      std::cout << "Thread " << id << " seed " << id+1 << ": " << result[id][i] << "\n";
      if(result[id][i] != expectedResult[id][i]) {
        std::cout << "but expected " << expectedResult[id][i] << "\n";
      }
    }
  }

  std::cout << "\nValues per seed must be equal\n";
  return 0;
}

11.04.2015

Local variable will not retain values between function invocations (Bug of the day 2)

Filed under: Allgemein — Tags: — Thomas @ 07:04

Lokale Variablen behalten ihren Wert NICHT zwischen Funktionsaufrufe!

Es gibt in diesem Beispiel keinen Compilerfehler und auch keinen von Fortran erkannten Laufzeitfehler.
Es gibt Compiler Warnings die diesen Fehler zeigen, aber dazu bedarf es mehr Code. Ich habe dieses Beispiel möglichst klein und übersichtlich gehalten.

Ausgabe ohne Optimierung und ohne Fehler

 kater@ktux:~$ gfortran -Wall -Wextra -fcheck=all  bugoftheday_openreadclose.f95 
kater@ktux:~$ ./a.out 
first round
DEBUG address of fhdl         BFC4D8FC
 open. set fhdl to         100
 read
 all file

 second round
DEBUG address of fhdl         BFC4D8FC
 open. set fhdl to         100
DEBUG address of test         BFC4D90C
 set test variabel to        1000
 read
 all file

Man beachte die unterschiedlichen Adressen der lokalen Variablen. Daher wird der Fehler nicht getriggert.

Ausgabe mit Optimierung und mit Fehler

kater@ktux:~$ gfortran -Wall -Wextra -fcheck=all  bugoftheday_openreadclose.f95 -O1
kater@ktux:~$ ./a.out  
 first round
DEBUG address of fhdl         BFA7DE38
 open. set fhdl to         100
 read
 all file

 second round
DEBUG address of fhdl         BFA7DE38
 open. set fhdl to         100
DEBUG address of test         BFA7DE38
 set test variabel to        1000
 read
 ERROR expect fhdl to be         100 but is        1000

Diesmal sind die Adressen der lokalen Variablen gleich! Die Variable fhdl wird ungeplant ueberschrieben!!
Welche Adresse die lokalen Variablen bekommen und ob so der Fehler auftritt oder nicht, ist zufällig und nicht beeinflussbar!!!


! Local variable will not retain values between function invocations
! compile: gfortran -Wall -Wextra -fcheck=all  bugoftheday_openreadclose.f95
! turn on any optimization to trigger the error


subroutine openread(action)
  use testmodule
  implicit none
  character(len=*), intent(in) :: action
  integer :: fhdl

  if(action == 'open') then
    fhdl = 100
    write(*,'(A,Z16)') "DEBUG address of fhdl ", loc(fhdl)
    write(*,*) "open. set fhdl to", fhdl
  else if(action == 'read') then
    write(*,*) "read"
    if(fhdl /= 100) then
      write(*,*) "ERROR expect fhdl to be", 100, "but is", fhdl
    else
      write(*,*) "all file"
    endif
  else if(action == 'close') then
      write(*,*) "close fhdl", fhdl
  endif
end subroutine


subroutine overwriteLocalValue()
  implicit none
  integer :: test

  test = 1000
  write(*,'(A,Z16)') "DEBUG address of test ", loc(test)
  write(*,*) "set test variabel to", test
end subroutine

program bugoftheday
  use testmodule
  implicit none

  write(*,*) "first round"
  call openread("open")
  call openread("read")

  write(*,*)
  write(*,*) "second round"
  call openread("open")
  call overwriteLocalValue
  call openread("read")
end program

10.04.2015

Bug of the day 1

Filed under: Allgemein — Tags: — Thomas @ 08:04

Der bug of the day kommt diesmal von mir!

Ich hab mir von eine Hand voll 3D Punkten neue Z Werte berechnet.
Aber hab den Z Wert im Punkt nie aktualisiert.
Und anstatt in der Ausgabedatei nun Bloedsinn steht oder lauter 0 Eintraege,
finden sich dort Werte die mit einen berechneten immer ein bis zwei Stellen am Anfang überein stimmen.
Fortran ist seltsam.
Ghosts updates mit einem leichten Fuzzy Faktor.
Da schwappen die Bits wohl von einen ins andere CPU Register über ;)

09.04.2015

Polymorphie virtual abstract pure deconstrutor WTF?

Filed under: Allgemein — Tags: — Thomas @ 08:04

Okay ich fasse mal zusammen. Ich sehe 3 Fälle, vom einfachsten zum schwersten.

1) Ganz normal Klassen ableiten.

class Base {
public:
  ~Base() {
    cout << "~TestClass\n";
  }
};

class Derived : public TestClass {
public:
  ~Derived() {
    cout << "~Derived\n";
  }
};

int main() {
  {
  cout << "a\n";
  Derived a;
  }
  cout << "b\n";
  Derived *b = new Derived();
  delete b;
}
Ausgabe:
a                                                                                                                
~Derived                                                                                                         
~Base
b
~Derived
~Base

Der Destructor wird immer aufgerufen.
2) Mit Polymorphie (Pointer ist vom Type Base)

class Base {
public:
  virtual ~Base() {
    cout << "~TestClass\n";
  }
};

class Derived : public TestClass {
public:
  virtual ~Derived() {
    cout << "~Derived\n";
  }
};

int main() {
  {
  cout << "a\n";
  Derived a;
  }
  cout << "b\n";
  Base *b = new Derived();
  delete b;
}
Ausgabe:
a                                                                                                                
~Derived                                                                                                         
~Base
b
~Derived
~Base

Der Destructor wird immer aufgerufen.

3) Mit Polymorphie und class Derived als abstracte Klasse


class Base {
public:
  virtual ~Base() = 0;
};

Base::~Base() {
    cout << "~Base\n";
}

class Derived : public Base {
public:
  virtual ~Derived() {
    cout << "~Derived\n";
  }
};

int main() {
  {
  cout << "a\n";
  Derived a;
  }
  cout << "b\n";
  Base *b = new Derived();
  delete b;
}
Ausgabe:
a                                                                                                                
~Derived                                                                                                         
~Base
b
~Derived
~Base

Der Destructor wird immer aufgerufen.
Der Destructor von Base ist zwar abstract, es gibt aber eine default Implementation, die im Zweifel garnichts macht. Es muss immer eine Implementation der Spezialfunktionen, wie dem Destructor, geben. Wird keine explicit angegeben, erstellt automatisch der Compiler eine. (Ausser man declariert ihn mit "=0" als abstract).

Wozu das ganze? Abstrace Klassen können nicht instanziiert werden, sie dienen wirklich nur als ein abstractes Interface. Sollte man es dennoch versuchen, bekommt man folgende Compilerfehler:

error: cannot declare variable ‘c’ to be of abstract type ‘Base’
 Base c;
 note:   because the following virtual functions are pure within ‘Base’:
 virtual ~Base() = 0;

PS: Wird das virtual keyword mal vergessen, wird der Base Destructor im ungünstigen Fall nicht aufgerufen ;)

Hf

Powered by WordPress