PPM + PUPMODES 3/7/13 = MESS

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

PPM + PUPMODES 3/7/13 = MESS

JakeSFR
The way PPM works in PUPMODES 3/7/13 can lead to unbelievable mess!

Some facts:
1. When installing a package, its files are copied directly to pup_ro1, but ~/.packages/package_name.files & ~/.packages/user-installed-packages are being created/updated in tmpfs (pup_rw) only
2. pinstall.sh can also do some stuff (eg. create symlinks) and it's done always at tmpfs level
3. Uninstallation process works on tmpfs level only, so it creates .wh files in pup_rw (until the session is saved)

This can cause some real PITA problems:

- after installing a package, but in case of crash/power failure/whatever, when session isn't saved, the package remains installed, but it's not easy to uninstall it, because stuff from ~/.packages/... hasn't been saved, hence it won't be listed on PPM's "Uninstall" list.

- installing package A, uninstalling it, then installing package B (which contains some files that A also had) and saving session will delete some of B's files.

- similarily, uninstalling a package and loading a .sfs, which contains some files which have just been uninstalled, will screen out those files from being visible in filesystem, unless (right after uninstalling) the session was saved _and_then_ .sfs was loaded
(...unless SAVEINTERVAL=0, in this case (un)loading an .sfs will just delete all .wh files from pup_rw, but this is sfs_load's/aufs' specific behavior/bug.)

- and so on, and so on...


In my opinion PPM should (un)install EVERYTHING directly into pup_ro1 or EVERYTHING into tmpfs. Not some files here and some files there.
For the sake of pinstall.sh (and because I have it this way all the time, so it's been well tested), I'd choose tmpfs...
The only downside of that choice is that when user is low on RAM and wants to install lots of (big) packages, will be forced to reboot from time to time, in order to flush stuff from pup_rw.

What you do think?

Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

01micko
Administrator
Pupmode saving routine to slow media in my mind is somewhat paranoid. It is a design of 10 years ago or more.

In that time 'slow media' (aka - flash or nand type media) have undergone vast improvements. Enough that even f2fs is somewhat paranoid, though optimisation should not be discouraged.

For now, go with your gut and commit the save to tmpfs.

That's my thought.. anyone else?
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

mavrothal
the simplest fix: run snapmerge after each (un)installation when pup_rw is in /tmpfs except in pupmode=5
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
In reply to this post by JakeSFR
the simplest fix: run snapmerge after each (un)installation when pup_rw is in /tmpfs except in pupmode=5
I had thought about this either, but I see two major downsides:

1. Some users (yup, me too) may not want all the stuff, that is in tmpfs ATM, to be automatically saved while (un)installing a .pet.
Besides, in PUPMODE=3/7/13 users should be guaranteed that everything remains in tmpfs, until (s)he explicitly decides to save the stuff or RAMSAVEINTERVAL has been set to do it periodically.
Snapmerge should be the one and only bridge between pup_rw and pup_ro1.
That was the primal concept of this modes, I believe.

Btw, I also hope Shinobar will consider my workaround for sfs_load:
http://www.murga-linux.com/puppy/viewtopic.php?p=783641#783641
because I just realized that this "fix" can be also used to replace current procedure, which searches for .wh files in pup_rw and removes corresponding items from pup_ro1 manually, before aufs gets remounted (remounting causes disappearance of all .wh files, hence the procedure).
So direct tampering with pup_ro1 would be no longer necessary also there.

2. Saving the session after each (un)installed pet would slow down the process terribly, even considering the latest improvements of snapmerge.


Ok, I'll push this change soon and we'll see if there will be any (negative) feedback from users of next alpha/beta release.

Thanks &
Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

mavrothal
> Some users may not want all the stuff, that is in tmpfs ATM, to be automatically saved.

I find the  idea that an OS is a toy rather secondary to dictate its behavior.
The fact that some modes use /tmpfs is only to minimize RW from slow/wear-prone  media not to make saving optional. Why an OS should have different behavior depending on the install media?

 
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
I understand your point, but I don't see, at the moment, any better solution, unfortunately.
The default state involves too much chaos, and the option to use snapmerge,  putting aside possible slowness, brings an element of unpredictability and inconsistency for those who chose RAMSAVEINTERVAL=0 and want to save only at shutdown, on demand or whatsoever.

The fact that some modes use /tmpfs is only to minimize RW from slow/wear-prone  media not to make saving optional.
Yeah, perhaps it (optional saving) was only a side effect, but it turned out to be really unique and appreciated, not only by me, feature of Puppy.

I respect your opinion though, so how about a compromise (again ;)?
Let's leave the decision to end users if they want to save directly to savefile or keep things in RAM.

Could be done in two ways, for example:
1. Ask every time while installing a package to save directly to savefile or tmpfs.
2. Additional (well explained) checkbox in configure.sh.

I prefer the second alternative, since I'm in the middle of reworking this script anyway, because in current state the config window is way too big, even for 800x600!
And this additional checkbox would perfectly fill some empty space on one of tabs, lol.

What do you say? :)

Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

q5sys
FWIW, the way puppy deals with save files and operating from tmpfs... is **the** reason I became interested in puppy, and the primary reason I refuse to completely convert over to Arch.

I like the persistance when I want it and non-persistance when I don't.  Obviously my opinion is of less importance than those who are doing the coding, just keep in mind this is one of those features that differientiates puppy from from every other linux distro.

-JT
(q5sys)
 

On June 18, 2014 2:24:19 PM EDT, "JakeSFR [via woof-CE]" <[hidden email]> wrote:

>
>
>I understand your point, but I don't see, at the moment, any better
>solution,
>unfortunately.
>The default state involves too much chaos, and the option to use
>snapmerge,
>putting aside possible slowness, brings an element of unpredictability
>and
>inconsistency for those who chose RAMSAVEINTERVAL=0 and want to save
>only at
>shutdown, on demand or whatsoever.


>Yeah, perhaps it (optional saving) was only a side effect, but it
>turned out
>to be really unique and appreciated, not only by me, feature of Puppy.
>
>I respect your opinion though, so how about a compromise (again ;)?
>Let's leave the decision to end users if they want to save directly to
>savefile or keep things in RAM.


--
Sent from Kaiten Mail. Please excuse my brevity.
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

mavrothal
In reply to this post by JakeSFR
> so how about a compromise (again ;)?

I'm always for choices (this is Linux after all ;) However, what I think is important is to have a consistent default state.
So if a checkbox appears in 3/7/13 modes offering an alternative option that's OK I believe. If it also checks that 1GB+ of RAM is installed before it appears, even better ;)
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
@Q5sys: yeah, that's also one of my main reasons, why I stick with Puppy!

@Mavrothal: Of course the checkbox (or maybe radiobuttons?) would be visible only in PUPMODE=3/7/13, but, I think, regardless of how much RAM is on the target system.

Anyway, I still hesitate how to set the default state:
1. checked (install to tmpfs)
2. uncheched (install to savefile)
3. initially checked, but only if RAM>=1GiB (rest is up to the user's choice)

The 3rd choice seems to be best, however it brings some indeterminacy, as user may wonder why on one box (>=1GiB) it's enabled and on another (<1GiB) disabled by default.
Besides, this 1GiB can't be an ultimate determinant - I have 4GiB of RAM, but If I had 2GiB, I'd still still need to reboot one time when (on freshly installed Puppy) I'm installing my set of usual pets.
And for some users (with e.g. 512MiB or even less) 2 or 3 small pets might be all they want, so it doesn't really matter if they'll go to tmpfs or savefile in such a case.
So perhaps it's better to choose no. 2, preserving the current state for now..?

Additionally, both modes would be described, e.g.:

"If you choose installing to tmpfs (memory), keep in mind that if you're low on RAM and want to install many large packages, all available memory may fill up quickly, so, in order to free it up, reboot will be necessary (saving the session is not enough)."

"If you choose installing directly to savefile, it's strongly recommended to save the session when you finish installing/uninstalling the package(s). It's because some important system files are still being created/updated/deleted only at tmpfs level."

(It's just a draft, so any corrections/improvements are really welcomed, as I'm not good in explaining things, even putting aside my poor English grammar)

Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

q5sys

I think #2 might be the 'safest' default for most regular users.  Let's keep in mind we want to keep puppy as simple as possible for users.  I'm all for giving users choices, but let's also not make them have to learn a crap-ton to figure out what to do. 
Aside from those of us who like to tweak and manipulate our systems, I think most users would just expect it to go to the save file so they have it on next boot.  they are unaware of -when- that happens, its just the behavior they are expecting.
the only reason we know better... is because we have far more knowledge about the way puppy works at a lower level.  So id say put the burden of effort (to click a check box to change the default) on those of us who know what that option actually means.
-q5sys



On June 18, 2014 6:41:56 PM EDT, "JakeSFR [via woof-CE]" <[hidden email]> wrote:
@Q5sys: yeah, that's also one of my main reasons, why I stick with Puppy!

@Mavrothal: Of course the checkbox (or maybe radiobuttons?) would be visible only in PUPMODE=3/7/13, but, I think, regardless of how much RAM is on the target system.

Anyway, I still hesitate how to set the default state:
1. checked (install to tmpfs)
2. uncheched (install to savefile)
3. initially checked, but only if RAM>=1GiB (rest is up to the user's choice)

The 3rd choice seems to be best, however it brings some indeterminacy, as user may wonder why on one box (>=1GiB) it's enabled and on another (<1GiB) disabled by default.
Besides, this 1GiB can't be an ultimate determinant - I have 4GiB of RAM, but If I had 2GiB, I'd still still need to reboot one time when (on freshly installed Puppy) I'm installing my set of usual pets.
And for some users, with e.g. 512MiB, 2 or 3 small pets are all they want, so it doesn't really matter if they'll go to tmpfs or savefile.
So perhaps it's better to choose no. 2, preserving the current state for now..?

Additionally, both modes would be described, e.g.:
"If you choose installing to tmpfs (memory), keep in mind that if you're low on RAM and want to install many large packages, all available memory may fill up quickly, so, in order to free it up, reboot will be necessary (saving the session is not enough)."
"If you choose installing directly to savefile, it's strongly recommended to save the session when you finish installing/uninstalling the package(s). It's because some important system files are still being created/updated/deleted only at tmpfs level."

(It's just a draft, so any corrections/improvements are really welcomed, as I'm not good in explaining things, even putting aside my poor English grammar)

Greetings!


If you reply to this email, your message will be added to the discussion below:
http://woof-ce.26403.n7.nabble.com/PPM-PUPMODES-3-7-13-MESS-tp538p547.html
To start a new topic under woof-CE, email [hidden email]
To unsubscribe from woof-CE, click here.
NAML

--
Sent from Kaiten Mail. Please excuse my brevity.
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

jamesbond3142
In reply to this post by JakeSFR
I'm a little late to this discussion.
As a comparison, Fatdog pet installer (silent_petget) *always* install to the topmost layer.
Before you say Fatdog64 has a different memory constraint - I do the same on FatdogArm too.
If you need more tmpfs space than you can run the equivalent of snapmerge in Fatdog and tmpfs is reclaimed.
If you want to install large packages *while not saving your session* then don't use packages, that's what SFS are for.
And if the SFS for packages you want to use isn't available (but the individual packages are), then use pet2sfs or something like that.

On Wed, 18 Jun 2014 15:41:56 -0700 (PDT)
"JakeSFR [via woof-CE]" <[hidden email]> wrote:

> "If you choose installing to tmpfs (memory), keep in mind that if you're low
> on RAM and want to install many large packages, all available memory may
> fill up quickly, so, in order to free it up, reboot will be necessary
> (saving the session is not enough)."

Why is the need for reboot? Won't snapmerge move all the files and remove them from tmpfs?

cheers!

James
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
This post was updated on .
In reply to this post by JakeSFR
Ok, so #2. Here's the sneak peak:



Any hints what could be corrected/clarified?

Fatdog pet installer (silent_petget) *always* install to the topmost layer.
@James: interesting, didn't know that. Were there any negative feedback from users about the way it works?

Why is the need for reboot? Won't snapmerge move all the files and remove them from tmpfs?
Nope, the "standard" snapmerge does not have this feature yet.
It's a long-term goal for me, though.

So far, I cooked this line (I'd like to avoid full lsof, which is not present by default, and busybox's lsof has crappy output):

find /proc/*/fd -type l -printf "%l\n" 2>/dev/null | grep '^/' | sort -u | grep -v '(deleted)$' > $TMPDIR/opened

Then, it would be a matter to find and delete some (not all!) files from pup_rw that are not present on the list created by the above line.
Can be done very quickly by:

cat $TMPDIR/opened $TMPDIR/opened $TMPDIR/dwelling | sort | uniq -u

('dwelling' is the list of files in pup_rw)

Btw, 'opened' x2 is a trick to make sure that ' | sort | uniq -u' will flush all of "opened" entries eventually.

That's only a concept so far, not tested on the living tissue, because I see some possible shortcomings and other caveats, so I don't think it will get included into snapmerge in the nearest future.

Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

jamesbond3142
On Thu, 19 Jun 2014 03:47:23 -0700 (PDT)
"JakeSFR [via woof-CE]" <[hidden email]> wrote:

>
> > Fatdog pet installer (silent_petget) *always* install to the topmost
> > layer.
>
> @James: interesting, didn't know that. Were there any negative feedback from
> users about the way it works?
>

None so far.
But Fatdog users tend to be quiet and stay in the background, unless they get very upset with video problems :)

cheers!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
In reply to this post by JakeSFR
Ok, pull request has been made: https://github.com/puppylinux-woof-CE/woof-CE/pull/477
Any final thoughts before I push it?
After small corrections looks like this:



Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

Karl Godt
Administrator
SFR, your screenshot indicates for me that you run enlightenment windowmanager using the " bling " theme .

If it is enlightenment WM --Did you install enlightenment via PPM ? And which version of e ?

 
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

bigpup
This post was updated on .
In reply to this post by JakeSFR
Stop using the term folder.
This is Linux.

The proper term is directory.

Try to make a folder in a Linux file manager.
Try to do it in the Puppy default file manager Rox.

You will not see an option to make a folder and nothing will be identified in detail view as a folder.
Visually it may look like a folder, but it is identified (type) as a directory.

It does not help anyone by not using the proper terms for things in Linux.

New users, and for that matter, everyone needs to use and learn proper Linux terms.
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

JakeSFR
In reply to this post by JakeSFR
@Bigpup: well, I used term 'savefolder' to avoid an inconsistency with this:



because I'm alergic to *inconsistency* much more than "folder" term. ;)
bigpup wrote
Try to make a folder in a Linux file manager.
Perfectly possible - see PcManFM & SpaceFM.

Karl Godt wrote
SFR, your screenshot indicates for me that you run enlightenment windowmanager using the " bling " theme .

If it is enlightenment WM --Did you install enlightenment via PPM ? And which version of e ?
Nah, it's Compiz/Emerald with "Ordinary Black Glass" theme (decorations)...but indeed, the GTK theme is "E17-GTK-gray".
I had some attempts with: http://slacke17.sourceforge.net/ but it never worked, unfortunately...
___________

Ok, changes have been merged (by Mick, thanks!).

Greetings!
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

Karl Godt
Administrator
In reply to this post by bigpup
Without *WORDS*

Is this really you, Bigpup ?

Or has someone hijacked your account ?
Reply | Threaded
Open this post in threaded view
|

Re: PPM + PUPMODES 3/7/13 = MESS

bigpup
Two wrongs do not make it right.

Sorry no one can understand why Linux terminology is important.

I guess I go back too far in remembering the early days of Linux and the dealings with Microsoft.

quote:
The term folder is used as a synonym for directory on the Microsoft Windows and Macintosh operating systems.

Only one of many examples:
http://www.linfo.org/directory.html

quote:
In GUIs on most operating systems, directories are typically represented by icons (i.e., small images) that resemble the manila folders that were formerly used in large numbers in most offices to organize paper documents.