Request for shutting down export script to old swgcraft.com

Discussion and suggestions regarding the site and/or forum.

Moderator: Forum Moderator

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Request for shutting down export script to old swgcraft.com

Post by Milla » Sun Sep 02, 2007 12:22 pm

Hello,

Today, I found that the export script on this site is probably responsible for messing up several entries in the swgcraft.com database.

Many resources that I've marked as available two days ago on Dantoone/FarStar on swgcraft.com vanished, and I had to reenter them: the old site indicated that they are existing resources entered WITHOUT STATS, although I've never done an update without entering ALL stats for the resources.

The recent changes section of the old site gave me no clue as to identity of the culprit (and the recent changes/recent unavailable pages are unavailable on this site), but since the Farstar resource section on this site is very badly synchronized to its the swgcraft.com counterpart, I must assume that the export/import script is doing a very sloppy job.

So I think, in order to protect the old site from bad updates, I suggest that the export script should be shut down for the time being. I'm going to update the old site until these quirks are removed, because the new site is still not as comfortable and functional as the old one - later I'll post a list of suggestions that could bring the new site on par.

A few mats that I suspect the export script messed up:

omnicogog,colat,517,273,714,754,322,367,716,752
dwinaud,kiirium,176,90,579,635,32,991,565,464
geptoras,titanium,71,395,145,298,399,980,330,318

Also, the export script have destroyed half of my flora entries on the swgcraft.com site on Corellia: the script marked them as swgcraft.co.uk entries, but without stats, and then made them unavailable, although they are still available on that planet.

Milara
Novice Crafter
Posts: 25
Joined: Sat Aug 18, 2007 10:47 am
Contact:

Post by Milara » Sun Sep 02, 2007 1:44 pm

I read a similar post to this last night:
http://www.swgcraft.co.uk/forums/viewtopic.php?t=457

It's sounding to me like there really is a bug hovering around here somewhere.

It's probably a rather simple one-- a check not properly triggering/filtering or whatnot, but something like this must be a high priority to look into. With the updates system here potentially buggy, it's essential that updating the legacy site not be impaired as well. I can't say I advocate shutting down synchronization all together but it's definitely necessary to correct this problem with all haste. If it can't be corrected immediately then the sync should be disabled until it can be corrected.

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 2:01 pm

Milara wrote:I read a similar post to this last night:
http://www.swgcraft.co.uk/forums/viewtopic.php?t=457

It's sounding to me like there really is a bug hovering around here somewhere.

It's probably a rather simple one-- a check not properly triggering/filtering or whatnot, but something like this must be a high priority to look into. With the updates system here potentially buggy, it's essential that updating the legacy site not be impaired as well. I can't say I advocate shutting down synchronization all together but it's definitely necessary to correct this problem with all haste. If it can't be corrected immediately then the sync should be disabled until it can be corrected.
What's the point of synchronizing if it messes up at least one of the sites, if not both?

Just a minute ago I found out that the swgcraft.co.uk script removed almost all active flora entries on Endor / FarStar, without even leaving a trace of its rampage in the Recent Changes section.

If the script is destructive and continues to operate unchecked, I see no point in updating any of these sites, rather I'll keep a personal database for myself.

The only other thing I can think of is that somebody removed all flora resources on this site, and the exporter did the same on the old site - since there is no recent changes/recent unavailable resources page on this site (coming soon.... ) a malevolent/careless updater can wreak havoc on the database.

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 3:56 pm

Further corruption caused by the script:

omifofef,lok fruit,560,696,892,966

when I tried to re-enter this resource, the old site reproduced the famous "vovoaide" copper resource bug that occurred here: the resource is in the database but it is not displayed on the relevant planet page.

User avatar
Sobuno
Developer
Posts: 2567
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Post by Sobuno » Sun Sep 02, 2007 3:59 pm

Disabled the despawn synchronization for SWGCraft.co.uk atm (Except for the "Remove after 30 days" thingy)

I've been trying to track this bug down for a couple of days now, I can't seem to reproduce it though:

I am finding a few bugs looking for it though, so I hope one of them eventually somehow is related to this

Can't see how it would change a resource to have no stats though, the 'Edit Resources' thingy doesn't edit the resource on .com

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 4:05 pm

One idea:

Recently, another FarStar player entered quite a few resources without stats on the development site.

It might be worth checking if an entry without stats could be a possible source of problems.

I'll try to create a snapshot of the next page where I find errors, and post the results here as I update it to the current status.

Also, it would be a great help if the Recent Changes / Recent Unavailable Resource pages were completed on this site - these pages may give a clue as to what went wrong during the update process.

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 4:37 pm

Just checked Naboo/FarStar:

swgcraft.com and swgcraft.co.uk are not in sync:

swgcraft.com :

Disofahiev, Lubricating Oil (Entered By quiet / Verified by MillaMerani)
DR=336, OQ=833 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)
Duroribine, Lubricating Oil (Entered By MillaMerani / Not Verified)
DR=212, OQ=264 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)
Eleoderck, Lubricating Oil (Entered By quiet / Verified by MillaMerani)
DR=732, OQ=193 (Edit Stats)
» Marked as available 2 days ago by MillaMerani (Unavailable / Remove)

- Polymer

Omnirabisis, Polymer (Entered By quiet / Not Verified)
DR=613, MA=973, OQ=964, SR=497, UT=971 *** (Edit Stats / Verify)
Vafaium, Polymer (Entered By MillaMerani / Verified by quiet)
DR=335, MA=661, OQ=817, SR=765, UT=432 (Edit Stats)

swgcraft.co.uk :

Eleoderck, Lubricating Oil (Entered By SWGCraft Parser / Not Verified) (Unavailable)
DR=732, OQ=193 (Edit Stats / Verify)
» Marked as available 2 days ago by SWGCraft Parser

- Polymer

Vafaium, Polymer (Entered By SWGCraft Parser / Not Verified) (Unavailable)
DR=335, MA=661, OQ=817, SR=765, UT=432 (Edit Stats / Verify)
» Marked as available 2 days ago by SWGCraft Parser

Also:

deghto,naboo fiber,978,123,60,266,675

was not present on both sites, although I entered it on swgcraft.com 5 days ago.

When I entered it again in the old site's mass input box, the old site said that it was an existing resource. Strange thing was that it marked resources type (naboo fiberplast), and all stat numbers bold, as if they had been ripped completely from the resource, and only the resource name had existed in the database.

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Sun Sep 02, 2007 4:41 pm

Milla wrote:What's the point of synchronizing if it messes up at least one of the sites, if not both?
Ahem, I believe I was one of the strongest voices to argue in favor of a two-way synchronization. Let me explain why.

Today there are numerous utilities, scripts, and applications out there that use the old, rusty SWGCraft.com as the site to pull data from. Since many of these cannot be changed in a snap (their developers may not even play SWG anymore) but rely on an up-to-date SWGCraft.com database. That is my one and only argument.

In fact all other arguments are in favor of a one-way synchronization, that this site just pulls data from SWGCraft.com. That would "force" crafters to come this way since it is only here they will find the most up-to-date and exhaustive stuff, including resources and schematics added to SWG more recently than Trokk was around. In due time I believe most valid stuff at SWGCraft.com will have found its way over here, just copied over or rewritten.

In hindsight, if turning off the two-way synch is the remedy to corrupted databases, and the inevitable cleaning work afterwards, then I cave in and would rather vote in favor of a one-way synchronization.

Kindly

/Zimoon

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Sun Sep 02, 2007 4:49 pm

But then we would be very happy to find an export feature at this site instead. Or am I blind? Is there some link to it that I miss out on?

/Z

User avatar
Sobuno
Developer
Posts: 2567
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Post by Sobuno » Sun Sep 02, 2007 4:52 pm

Zimoon wrote:But then we would be very happy to find an export feature at this site instead. Or am I blind? Is there some link to it that I miss out on?

/Z
http://www.swgcraft.co.uk/forums/viewtopic.php?t=192

User avatar
Zimoon
Forum Moderator
Posts: 4817
Joined: Mon May 14, 2007 6:55 am
Location: Stockholm, SE
Contact:

Post by Zimoon » Sun Sep 02, 2007 5:09 pm

I had a vague memory of me seeing them, I must have developed some kindof Alzheimer I guess :(

Thanks

/Z

User avatar
Sobuno
Developer
Posts: 2567
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Post by Sobuno » Sun Sep 02, 2007 5:20 pm

The time of the despawn by SWGCraft Parser suggest that the importer made the despawning of [res]Farstar:Disofahiev[/res]. This will only happen if the script cannot find the specific resource in the file fetched from SWGCraft.com

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 6:44 pm

On swgcraft.com:

Flora Resources - Mark As Completed
MillaMerani marked as completed 2 days ago

- Evergreen Wood

Rogiga, Tatooinian Evergreen Wood (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

- Deciduous Wood

Eocoflec, Tatooinian Deciduous Wood (Entered By MillaMerani / Verified by quiet)
DR=654, MA=517, OQ=677, SR=346, UT=5 (Edit Stats)
» Marked as available 17 days ago by MillaMerani (Unavailable / Remove)

- Flowers

Iobled, Tatooinian Flower Fruit (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

- Tubers

Idao%, Tatooinian Vegetable Tubers (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

- Beans

Sejoho, Tatooinian Vegetable Beans (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

- Domesticated Wheat

Kevor, Tatooinian Domesticated Wheat (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

- Domesticated Rice

Socemat, Tatooinian Domesticated Rice (Entered By SWGCraft.co.uk / Not Verified)
DR=59, FL=681, PE=308, OQ=43 (Edit Stats / Verify)
» Marked as available 12 days ago by SWGCraft.co.uk (Unavailable)

On swgcraft.co.uk

Flora Resource

- Domesticated Rice

Socemat, Tatooinian Domesticated Rice (Entered By Milla / Not Verified) (Unavailable)
DR=59, FL=681, PE=308, OQ=43 (Edit Stats)
» Marked as available 12 days ago by Milla (Remove)


- Domesticated Wheat

Kevor, Tatooinian Domesticated Wheat (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay


- Flowers

Iobled, Tatooinian Flower Fruit (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay


- Beans

Sejoho, Tatooinian Vegetable Beans (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay


- Tubers

Idao%&$@iet, Tatooinian Vegetable Tubers (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay


Idao%, Tatooinian Vegetable Tubers (Entered By SWGCraft Parser / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by SWGCraft Parser


- Deciduous Wood

Eocoflec, Tatooinian Deciduous Wood (Entered By SWGCraft Parser / Not Verified) (Unavailable)
DR=654, MA=517, OQ=677, SR=346, UT=5 (Edit Stats / Verify)
» Marked as available 17 days ago by SWGCraft Parser


- Evergreen Soft Wood

Rogiga, Tatooinian Evergreen Wood (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay

Missing resources:

Swgcraft.com indicated these changes:

Rogiga Tatooinian Evergreen Wood Available SWGCraft.co.uk 5hrs ago
Iobled Tatooinian Flower Fruit Available SWGCraft.co.uk 5hrs ago
Eigri Tatooinian Flower Fruit Unavailable SWGCraft.co.uk 5hrs ago
Athetor Tatooinian Evergreen Wood Unavailable SWGCraft.co.uk 5hrs ago
Idao% Tatooinian Vegetable Tubers Available SWGCraft.co.uk 5hrs ago
Idaotwatiet Tatooinian Vegetable Tubers Unavailable SWGCraft.co.uk 5hrs ago
Upikeri Tatooinian Vegetable Beans Unavailable SWGCraft.co.uk 5hrs ago
Eupou Tatooinian Domesticated Wheat Unavailable SWGCraft.co.uk 5hrs ago
Kevor Tatooinian Domesticated Wheat Available SWGCraft.co.uk 5hrs ago
Sejoho Tatooinian Vegetable Beans Available SWGCraft.co.uk 5hrs ago

Flora Resources - Mark As Completed
MillaMerani marked as completed 0 minutes ago

- Evergreen Wood

Rogiga, Tatooinian Evergreen Wood (Entered By MillaMerani / Not Verified)
DR=72, MA=863, OQ=442, SR=343, UT=137 (Edit Stats)
» Marked as available 6 hours ago by SWGCraft.co.uk (Unavailable)

- Conifer Wood

Asobiarl, Tatooinian Conifer Wood (Entered By MillaMerani / Not Verified)
DR=12, MA=628, OQ=59, SR=373, UT=234 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)

- Deciduous Wood

Eocoflec, Tatooinian Deciduous Wood (Entered By MillaMerani / Verified by quiet)
DR=654, MA=517, OQ=677, SR=346, UT=5 (Edit Stats)
» Marked as available 17 days ago by MillaMerani (Unavailable / Remove)

- Flowers

Iobled, Tatooinian Flower Fruit (Entered By MillaMerani / Not Verified)
DR=142, FL=712, PE=53, OQ=134 (Edit Stats)
» Marked as available 6 hours ago by SWGCraft.co.uk (Unavailable)

- Berries

Awococlo, Tatooinian Berry Fruit (Entered By MillaMerani / Not Verified)
DR=64, FL=736, PE=389, OQ=81 (Edit Stats)
» Marked as available 4 days ago by MillaMerani (Unavailable / Remove)

- Fruits

Elvafeleu, Tatooinian Fruit (Entered By MillaMerani / Not Verified)
DR=390, FL=773, PE=143, OQ=280 (Edit Stats)
» Marked as available 4 days ago by MillaMerani (Unavailable / Remove)

- Fungi

Allado, Tatooinian Vegetable Fungus (Entered By MillaMerani / Not Verified)
DR=551, FL=561, PE=655, OQ=660 (Edit Stats)
» Marked as available 17 days ago by MillaMerani (Unavailable / Remove)

- Tubers

Idaotwatiet, Tatooinian Vegetable Tubers (Entered By MillaMerani / Not Verified)
DR=9, FL=761, PE=140, OQ=81 (Edit Stats)
» Marked as available 0 minutes ago by MillaMerani (Unavailable / Remove)

- Beans

Sejoho, Tatooinian Vegetable Beans (Entered By MillaMerani / Not Verified)
DR=775, FL=991, PE=183, OQ=961 ** (Edit Stats)
» Marked as available 6 hours ago by SWGCraft.co.uk (Unavailable)

- Greens

Aobono, Tatooinian Vegetable Greens (Entered By MillaMerani / Not Verified)
DR=194, FL=788, PE=75, OQ=856 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)

- Wild Wheat

Fipokteo, Tatooinian Wild Wheat (Entered By MillaMerani / Not Verified)
DR=335, FL=771, PE=33, OQ=614 (Edit Stats)
» Marked as available 2 days ago by MillaMerani (Unavailable / Remove)

- Domesticated Wheat

Kevor, Tatooinian Domesticated Wheat (Entered By MillaMerani / Not Verified)
DR=552, FL=337, PE=849, OQ=604 (Edit Stats)
» Marked as available 6 hours ago by SWGCraft.co.uk (Unavailable)

- Wild Oats

Posti, Tatooinian Wild Oats (Entered By MillaMerani / Not Verified)
DR=694, FL=882, PE=521, OQ=91 (Edit Stats)
» Marked as available 2 days ago by MillaMerani (Unavailable / Remove)

- Domesticated Oats

Ekliov, Tatooinian Domesticated Oats (Entered By MillaMerani / Not Verified)
DR=126, FL=499, PE=775, OQ=323 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)

- Wild Rice

Aotta, Tatooinian Wild Rice (Entered By MillaMerani / Not Verified)
DR=261, FL=799, PE=199, OQ=668 (Edit Stats)
» Marked as available 5 days ago by MillaMerani (Unavailable / Remove)

- Domesticated Rice

Socemat, Tatooinian Domesticated Rice (Entered By SWGCraft.co.uk / Verified by MillaMerani)
DR=59, FL=681, PE=308, OQ=43 (Edit Stats)
» Marked as available 12 days ago by SWGCraft.co.uk (Unavailable)

- Wild Corn

Echen, Tatooinian Wild Corn (Entered By MillaMerani / Not Verified)
DR=561, FL=646, PE=575, OQ=513 (Edit Stats)
» Marked as available 2 days ago by MillaMerani (Unavailable / Remove)

- Domesticated Corn

Evauka, Tatooinian Domesticated Corn (Entered By MillaMerani / Not Verified)
DR=700, FL=611, PE=884, OQ=304 (Edit Stats)
» Marked as available 2 days ago by MillaMerani (Unavailable / Remove)


You can see a strange resource name corruption here:

On swgcraft.com
Idao%, Tatooinian Vegetable Tubers (Entered By SWGCraft.co.uk / Not Verified)
No stats available for this resource (Edit Stats)
» Marked as available 5 hours ago by SWGCraft.co.uk (Unavailable)

On swgcraft.co.uk
Idao%&$@iet, Tatooinian Vegetable Tubers (Entered By Quay / Not Verified) (Unavailable)
No stats available for this resource (Edit Stats)
» Marked as available 6 hours ago by Quay

The correct entry was made by myself on 29th August, 2007 (only my records are available as the resource update has changed the time and date entry):

Idaotwatiet, Tatooinian Vegetable Tubers (Entered By MillaMerani / Not Verified)
DR=9, FL=761, PE=140, OQ=81 (Edit Stats)
» Marked as available 0 minutes ago by MillaMerani (Unavailable / Remove)

However, the Recent Changes entries on swgcraft.com did not reflect the actual changes on the site, as a lot more resources were made unavailable compared to the listing.

Milla
Apprentice Crafter
Posts: 77
Joined: Thu Jul 12, 2007 4:01 pm

Post by Milla » Sun Sep 02, 2007 6:51 pm

Another entry that was deleted by the script or somebody else:

nodaieium,hardened arvhesium,5,714,295,859,539,198,569,234

I added this resource to swgcraft.com 2 days ago (30th August), marking it available on Tatooine.

It is not available on swgcraft.co.uk at the moment.

EDIT: I tried to add the resources to swgcraft.co.uk and succeeded. Apparently, the script successfuly imported this entry form swgcraft.com two days ago. Then something happened to it during the last two days.

Also, it seems the mass resource entry window has serious trouble with seemingly normal text input:

swgcraft_start
nodaieium,hardened,5,714,295,859,539,198,569,234
swgcraft_end

was broken up into

swgcraft_start
nodaieum
hardened
5
714
etc.

I tried to delete and re-type enters at the end of swgcraft_start and swgcraft_com, and also at the end of the resource data, but the site only accepted it when I copied the text into windows notepad, deleted the Enter characters, retyped them again, and then copied and pasted the text into the mass input box.

User avatar
Sobuno
Developer
Posts: 2567
Joined: Sun Mar 25, 2007 2:17 am
Contact:

Post by Sobuno » Sun Sep 02, 2007 7:51 pm

Milla wrote: Also, it seems the mass resource entry window has serious trouble with seemingly normal text input:

swgcraft_start
nodaieium,hardened,5,714,295,859,539,198,569,234
swgcraft_end
A space after swgcraft_end will cause this, I just tested the above in FF and IE and it works

Post Reply

Who is online

Users browsing this forum: No registered users and 7 guests