Ruby Forum Ruby-core > Severe problem with garbage collection

Hi,

I have a really large amount of data here and a Ruby program
dealing with it. I noticed how this program started to use swap
space and then, hours later, terminated. Then I watched (Linux
here) "/proc/$$/statm". Yes, the whole memory was eaten up.

I know that my program does not have a bug because it is running
for a year now without complaint. So I downgraded my Ruby from
1.8.6_p230 to 1.8.6_p111 and--voila--the statm output yields
constant values and the program finishes successfully.

Sorry, I cannot provide any data because it is from a lawyers
office and underlies duty of confidentially. I would search on my
own for the bug if someone could give me a hint where to start
doing this.

Thanks in advance.

Bertram

On Aug 11, 2008, at 5:39 AM, Bertram Scharpf wrote:

> constant values and the program finishes successfully.
>
> Sorry, I cannot provide any data because it is from a lawyers
> office and underlies duty of confidentially. I would search on my
> own for the bug if someone could give me a hint where to start
> doing this.

There were several serious problems with 1.8.6-p230. Please consider
upgrading to the latest patch release which is 1.8.6-p287 which is
available from ruby-lang.org.

cr

[1] http://www.ruby-lang.org/en/

Bertram Scharpf wrote:
> constant values and the program finishes successfully.
> 
> Sorry, I cannot provide any data because it is from a lawyers
> office and underlies duty of confidentially. I would search on my
> own for the bug if someone could give me a hint where to start
> doing this.

Hi,

I'd start with looking at the changes between 1.8.6_p230 and _p111 to
file gc.c. Should be pretty easy to extract those changes using "svn
diff".

You could also compare the number of allocated objects (using class
ObjectSpace) for both Ruby versions when your program is running. Maybe
there is an anormaly.

You could as well try your program with Ruby 1.9? Maybe it runs. Maybe
it exhibits the same problem.

Or do a binary search over the Ruby versions. Try 1.8.6_p220. If it
works, try 1.8.6_p225, until you find the version which introduced the
"bug". Then use diff to track down the bug even further.

Regards,

   Michael

On Mon, Aug 11, 2008 at 3:39 AM, Bertram Scharpf
<lists@bertram-scharpf.de> wrote:
> I have a really large amount of data here and a Ruby program
> dealing with it. I noticed how this program started to use swap
> space and then, hours later, terminated. Then I watched (Linux
> here) "/proc/$$/statm". Yes, the whole memory was eaten up.

Thanks for reporting the issue. The 1.8.6_p230 official release had a
memory leak in it, which is probably what you encountered. That
problem has been resolved:
http://redmine.ruby-lang.org/issues/show/216

-igal

Hi,

Am Dienstag, 12. Aug 2008, 01:51:57 +0900 schrieb Igal Koshevoy:
> http://redmine.ruby-lang.org/issues/show/216
Arrgh! About two years ago I happened to unmask the Gentoo package on
the machine in question and I didn't remember that because I mainly use
FreeBSD meanwhile. Sorry for the noise. As I noticed today, the p230
release disappeared from the Gentoo portage tree.

I had a short look at gc.c of the versions p111, p230, and p287. I
didn't find anything that's obvious.

I just tried some pieces of code like the ones I cite below. Sorry, I
seem to have too less understandig when garbage collection takes place
to examine this further for now.

Bertram


________________________________

  def statm
    File.open "/proc/#$$/statm" do |s| s.read end
  end

  def f
    a = []
    1000.times { a.push yield }
    puts statm
  end
  f { "hello "*4096 }
  puts statm

  def g
    a = "hello "*4096
    g
  end
  begin
    g
  rescue
    puts statm
  end
  puts statm