Discussion:
[Announcement] First public git repo of the complete AUR.
(too old to reply)
Thomas Dziedzic
2011-02-06 19:31:06 UTC
Permalink
Hey there Archers,
I decided to work on this little project over the weekend.
It is a complete git clone of the AUR from the source and it will be
updated regularly (at least once a day) though probably as frequent as
30minutes (please see `Side Notes' for explanation).
What this means is that history for the AUR will now be possible!
Also, for those of you wanting to have an abs like tree for all of the
aur, it is now possible.
I hope this opens up new opportunities, and new possibilities for everyone.

To get the complete git repo, run:
git clone git://pkgbuild.com/aur.git

The web interface is at:
http://pkgbuild.com/git/aur.git/

Also, please take a look at the official readme at:
http://pkgbuild.com/~td123/readme

Special Thanks to Ioni for enabling the git server & Bluewind for
setting up the web interface.

Cheers!
-Thomas Dziedzic

Side Note:
Although it will be updated at least once a day, it will most likely
be updated more frequently (around 30 minute intervals with a script),
I just can't guarantee that for now.
Work is ongoing to automate the sync with the aur tree, but at this
moment, there is no secure way of doing this.
Automating the sync will guarantee the intervals between syncs, I am
aiming for around 30 minute to 3 hour intervals between syncs.
A final goal might be to get continuous updates. Enjoy! =)
Lukáš Jirkovský
2011-02-06 19:33:13 UTC
Permalink
Post by Thomas Dziedzic
Hey there Archers,
I decided to work on this little project over the weekend.
It is a complete git clone of the AUR from the source and it will be
updated regularly (at least once a day) though probably as frequent as
30minutes (please see `Side Notes' for explanation).
What this means is that history for the AUR will now be possible!
Also, for those of you wanting to have an abs like tree for all of the
aur, it is now possible.
I hope this opens up new opportunities, and new possibilities for everyone.
git clone git://pkgbuild.com/aur.git
http://pkgbuild.com/git/aur.git/
http://pkgbuild.com/~td123/readme
Special Thanks to Ioni for enabling the git server & Bluewind for
setting up the web interface.
Cheers!
-Thomas Dziedzic
Although it will be updated at least once a day, it will most likely
be updated more frequently (around 30 minute intervals with a script),
I just can't guarantee that for now.
Work is ongoing to automate the sync with the aur tree, but at this
moment, there is no secure way of doing this.
Automating the sync will guarantee the intervals between syncs, I am
aiming for around 30 minute to 3 hour intervals between syncs.
A final goal might be to get continuous updates. Enjoy! =)
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
Lukáš Jirkovský
2011-02-06 19:35:22 UTC
Permalink
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
It seems there's more:
apercu.tgz
Thomas Dziedzic
2011-02-06 19:38:24 UTC
Permalink
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will
have to deal with it.
Thomas Dziedzic
2011-02-06 19:39:07 UTC
Permalink
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will
have to deal with it.
I will try to clean up my rules this evening.

Thanks for your suggestions :)
Thomas S Hatch
2011-02-06 20:10:43 UTC
Permalink
Post by Thomas Dziedzic
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Yes, I know, the aur tree isn't completely clean, for now, you will
have to deal with it.
I will try to clean up my rules this evening.
Thanks for your suggestions :)
This is awesome, thanks!
Bernardo Barros
2011-02-06 20:15:38 UTC
Permalink
Very good idea, I will check it out.

AUR was under some kind of version control before or it never was?
Thomas Dziedzic
2011-02-06 20:26:43 UTC
Permalink
On Sun, Feb 6, 2011 at 2:15 PM, Bernardo Barros
Post by Bernardo Barros
Very good idea, I will check it out.
AUR was under some kind of version control before or it never was?
I think it was never.
Thomas Dziedzic
2011-02-06 20:34:43 UTC
Permalink
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to
be done remotely to do it properly.

Somebody ran a clean up script as evidenced by this commit :P
http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b2061db2c34

The only leftover files are the .tgz files which will also have to get
cleaned up in the main aur tree. Thanks again!
Loui Chang
2011-02-06 22:43:59 UTC
Permalink
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to
be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P
http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b2061db2c34
That was me hehe. I was curious to see what would happen.
So it looks like you're just copying over what's in the filesystem.
It might be a good idea to work with aur-dev to help make the source
cleaner, then everyone can benefit - even those who don't use your git
repo.

You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
keenerd
2011-02-06 22:58:57 UTC
Permalink
Post by Loui Chang
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
Hey, you are stealing my idea! :-) AUR3 does that, and it saves
several hundred megabytes. Completely worth it.

-Kyle
http://kmkeen.com
Thomas Dziedzic
2011-02-06 23:52:05 UTC
Permalink
Post by Loui Chang
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
Hey, you are stealing my idea!  :-)  AUR3 does that, and it saves
several hundred megabytes.  Completely worth it.
-Kyle
http://kmkeen.com
I fail to see how this is worth it, imo, a better system is to convert
to git and not track the src.tar.gz

Is there a good reason for this switch? To save 450mb is not a good
reason imo, for an incomplete listing of all the files.

-Thomas
Loui Chang
2011-02-07 00:22:56 UTC
Permalink
Post by Thomas Dziedzic
Post by Loui Chang
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
Hey, you are stealing my idea!  :-)  AUR3 does that, and it saves
several hundred megabytes.  Completely worth it.
I fail to see how this is worth it, imo, a better system is to convert
to git and not track the src.tar.gz
Is there a good reason for this switch? To save 450mb is not a good
reason imo, for an incomplete listing of all the files.
Well, there are several reasons. Lukas' commit message from commit ec0dfc2
briefly summarizes it.
Post by Thomas Dziedzic
Automatic tarball extraction was vulnerable in different ways. Users
should also only use source tarballs to build packages, so this has
been removed completely. From now on, only the PKGBUILD is extracted
in a secure manner.
Also,

I'm not really sure that git is the best way to distribute source
packages, but I'm glad that you're exploring different options. :D

If I want to obtain or share a few build scripts for a few packages I
really don't want to keep a 450mb repo.

I have heard about shallow checkouts being implemented in git though, so
maybe it could work. devtools uses subversion at least partially because
of this large checkout issue.
Thomas Dziedzic
2011-02-07 15:21:25 UTC
Permalink
Post by Loui Chang
Post by Thomas Dziedzic
Post by Loui Chang
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
Hey, you are stealing my idea!  :-)  AUR3 does that, and it saves
several hundred megabytes.  Completely worth it.
I fail to see how this is worth it, imo, a better system is to convert
to git and not track the src.tar.gz
Is there a good reason for this switch? To save 450mb is not a good
reason imo, for an incomplete listing of all the files.
Well, there are several reasons. Lukas' commit message from commit ec0dfc2
briefly summarizes it.
Post by Thomas Dziedzic
Automatic tarball extraction was vulnerable in different ways. Users
should also only use source tarballs to build packages, so this has
been removed completely. From now on, only the PKGBUILD is extracted
in a secure manner.
Also,
I'm not really sure that git is the best way to distribute source
packages, but I'm glad that you're exploring different options. :D
If I want to obtain or share a few build scripts for a few packages I
really don't want to keep a 450mb repo.
I have heard about shallow checkouts being implemented in git though, so
maybe it could work. devtools uses subversion at least partially because
of this large checkout issue.
Well, whatever the case, I stayed up a while working on finishing this
new version of the system.
The new version syncs only the .tar.gz files, which makes it less
resource hungry, and this also adds a natural ability to commit per
packages :)
I still need to test out deleting functionality, so if you could run
the cleanup script at some point today, that would be awesome.

Cheers!

Thomas Dziedzic
2011-02-06 23:19:23 UTC
Permalink
Post by Loui Chang
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to
be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P
http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b2061db2c34
That was me hehe. I was curious to see what would happen.
So it looks like you're just copying over what's in the filesystem.
It might be a good idea to work with aur-dev to help make the source
cleaner, then everyone can benefit - even those who don't use your git
repo.
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
Thomas Dziedzic
2011-02-06 23:42:41 UTC
Permalink
Post by Thomas Dziedzic
Post by Loui Chang
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to
be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P
http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b2061db2c34
That was me hehe. I was curious to see what would happen.
So it looks like you're just copying over what's in the filesystem.
It might be a good idea to work with aur-dev to help make the source
cleaner, then everyone can benefit - even those who don't use your git
repo.
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
Nvm, I read your mail wrong.
Loui Chang
2011-02-06 23:53:44 UTC
Permalink
Post by Thomas Dziedzic
Post by Loui Chang
Post by Thomas Dziedzic
Post by Lukáš Jirkovský
Post by Lukáš Jirkovský
What's this?
http://pkgbuild.com/git/aur.git/tree/PKGBUILD
apercu.tgz
Since this clean up is more of an aur tree issue, not mine, it has to
be done remotely to do it properly.
Somebody ran a clean up script as evidenced by this commit :P
http://pkgbuild.com/git/aur.git/commit/?id=19c10dddf1c78cb14f958e02cd6a4b2061db2c34
That was me hehe. I was curious to see what would happen.
So it looks like you're just copying over what's in the filesystem.
It might be a good idea to work with aur-dev to help make the source
cleaner, then everyone can benefit - even those who don't use your git
repo.
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
So PKGBUILDs aren't going to be able to be viewed?
Yes they will, but not the other files.
Pierre Chapuis
2011-02-07 12:55:27 UTC
Permalink
Post by Loui Chang
You probably want to grab the tarballs, and extract what's in those.
The next release of the AUR will only have tarballs and PKGBUILDs.
The other files won't be extracted.
Would it be possible to also extract .install scripts? I usually
read those before installing AUR packages since they don't come
from a trusted source.
--
Pierre 'catwell' Chapuis
keenerd
2011-02-06 22:42:45 UTC
Permalink
Post by Thomas Dziedzic
Hey there Archers,
I decided to work on this little project over the weekend.
It is a complete git clone of the AUR from the source and it will be
updated regularly (at least once a day) though probably as frequent as
30minutes (please see `Side Notes' for explanation).
What this means is that history for the AUR will now be possible!
Also, for those of you wanting to have an abs like tree for all of the
aur, it is now possible.
I hope this opens up new opportunities, and new possibilities for everyone.
This is awesome. Glad to see some official support for such crazy
stuff. The change sets look much more impressive than I thought they
would. There is still the Hard Problem of getting write access
control for 20k accounts and grafting a comment system onto git (both
of which were really discouraging when I tried solving it) but today
is a huge win for CLI usability in Arch.

I am also glad to see it done, as now version tracking can be removed
from the list of silly-things-for-AUR3-to-try as there is no reason
for me to duplicate efforts :-)

-Kyle
http://kmkeen.com
Continue reading on narkive:
Loading...