Avian Gamers Network

Forum
It is currently Fri May 02, 2025 4:59 pm

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: xml.gz parsing in PHP
PostPosted: Thu Jan 01, 2004 11:27 pm 
Offline
Site Admin
User avatar

Joined: Mon Jul 01, 2002 4:33 am
Posts: 6698
Location: Silver Spring, MD
Now that I have some free time I'm back trying to work on my harvester managment program.

Its basically done but I need to get a complete database of resources for it to run smoother.
swgcraft.com has this file
http://www.swgcraft.com/sendfile.php?file=currentresources.xml.gz

I would like a way to get this info into a php db. I have no experience with xml so I have no clue on how to even begin.
I dont want to use the updates of this file I just want to parse it into my db for my use.

anyone know how this can be done or where to begin?
I am running PHP/apache

_________________
Moge


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 02, 2004 12:31 am 
Offline
Site Admin
User avatar

Joined: Mon Jul 01, 2002 4:33 am
Posts: 6698
Location: Silver Spring, MD
Ok I found this code and it works well for dislaying the data. Now I just need to be able to add it to my mySQL database

[code]
<?php

// Do not alter these global variables
$curTag = "";
$serverName = "";
$planetName = "";
$resourceData = array();


// Customize this method to update your database. You will probably want to replace the
// sample code here with a series of SQL statements customized for your database.
function handleResource() {
global $resourceData, $serverName, $planetName;
// Remove this "if" statment if you want to process all servers, or change
// it to match the single server you are interested in.
//if($serverName == "Chilastra") {
$availTime = $resourceData["AVAILABLE_TIMESTAMP"];
$output = sprintf("%s - %s : CD = %d, CR = %d, DR = %d, ER = %d, FL = %d, HR = %d, " .
"MA = %d, OQ = %d, PE = %d, SR = %d, UT = %d, Available on: %s<BR>",
$resourceData["NAME"],
$resourceData["TYPE"],
$resourceData["CD"],
$resourceData["CR"],
$resourceData["DR"],
$resourceData["ER"],
$resourceData["FL"],
$resourceData["HR"],
$resourceData["MA"],
$resourceData["OQ"],
$resourceData["PE"],
$resourceData["SR"],
$resourceData["UT"],
date('Y-m-d', $availTime));
echo($output);
}
//}



// Clears the $resourceData array.
// DO NOT MODIFY THIS FUNCTION
function clearResourceData() {
global $resourceData;
$resourceData["NAME"] = "";
$resourceData["CD"] = 0;
$resourceData["CR"] = 0;
$resourceData["DR"] = 0;
$resourceData["ER"] = 0;
$resourceData["FL"] = 0;
$resourceData["HR"] = 0;
$resourceData["MA"] = 0;
$resourceData["OQ"] = 0;
$resourceData["PE"] = 0;
$resourceData["SR"] = 0;
$resourceData["UT"] = 0;
}

// DO NOT MODIFY
function startElement($parser, $elementName, $elementAttrs) {
global $serverName, $planetName, $curTag;
$curTag = $elementName;

switch($elementName) {
case "SERVER":
$serverName = $elementAttrs["NAME"];
break;

case "PLANET":
$planetName = $elementAttrs["NAME"];
break;
}
}

// We have received an ending tag.
// DO NOT MODIFY
function endElement($parser, $elementName) {
switch($elementName) {
case "RESOURCE":
// We have gathered all of the information for this resource.
handleResource();
clearResourceData();
break;
}
}


// This is regular character data. We are only interested in it if
// the current tag is RESOURCE.
// DO NOT MODIFY
function characterData($parser, $data) {
global $resourceData, $curTag;

$data = trim($data);

if(strlen($data) > 0 && $data != " "):
$resourceData[$curTag] = $data;
endif;
}


// Connect to the SWGCraft.com site, open the resource file and parse the info.
// DO NOT MODIFY
function parseResourceXML() {
// Start of the main body of code
echo("Opening the XML file...<BR>\n");
$parser = xml_parser_create();
xml_set_element_handler($parser, 'startElement', 'endElement');
xml_set_character_data_handler($parser, 'characterData');
//$fp = gzopen('http://www.swgcraft.com/static/resources/currentresources.xml.gz', 'r') or die("<p>Cannot open the XML source file");
$fp = gzopen('currentresources.xml.gz', 'r') or die("<p>Cannot open the XML source file");

echo("Parsing the XML file...\n");
while($data = gzread($fp, 4096)) {
$result = xml_parse($parser, $data, gzeof($fp)); // or die("XML ERROR");
if(!$result) {
$errMsg = xml_error_string(xml_get_error_code($parser));
$lineNum = xml_get_current_line_number($parser);
$colNum = xml_get_current_column_number($parser);
echo($errMsg . " on line $lineNum, column $colNum\n");
die("Error parsing the file.");
}
}

// Close the XML file
gzclose($fp);
xml_parser_free($parser);
}
?>
<HTML>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<!-- Modify to use your stylesheet, if you use one at all -->
<LINK REL="stylesheet" TYPE="text/css" HREF="stylesheet.css">
<title>Update Resources</title>
</head>
<BODY>
<p>Updating resource information from SWGCraft.com</p>
<?php
parseResourceXML();
?>
<p>Resources have been updated!</p>
</BODY>

_________________
Moge


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 02, 2004 1:08 pm 
Offline
User avatar

Joined: Tue Mar 04, 2003 3:14 am
Posts: 1408
Location: Atlanta, GA
What? Huh? Good luck with that.

_________________
Retired Weaponsmith


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 03, 2004 12:01 am 
Offline
Site Admin
User avatar

Joined: Mon Jul 01, 2002 4:33 am
Posts: 6698
Location: Silver Spring, MD
For those interested, this is what I came up with. again I know very very little about XML so all the XML code is ripped from a display script that I morphed into an input script.
It is quite clunky and slow but I managed to get all the Resource names and resource Types into my mysql db.

there are:
530 resource Types
6583 resource names

I am sure Im missing some but I will rerun the scrpit weekly from the swgcraft xml export.

Now the hard part. I need to link the resource trees together. the XML code only give Types and names eg Corellian Animal Bone Miwhe
I now need backwards link Corellian Animal bone to: Bone, Flora Structural, Organic as these are not given in the XML.

To make this harder different resources have different depths. Animal Bone has a depth of 6 Organic, Flora Structural, Bone, Corellian Animal Bone, Miwhe.
where say Beyrllius Copper has a depth of 7.

Im tring to come up with a structure so all resources have the same depth but I think I need to bite the bullet and make different depth trees for each resource general type (cant to one for all of minerals becuase LGO, metal have 7 where Gemstone has 6 etc). Then figure out a way to make these trees searchable as some trees will have the same information. The tree that holds LGO & Metal will have Inorganic, Mineral as will the Gemstone tree.

But thats a problem for another day.


NOTE: Some of the code in input.php I commented out as it is left unused from the origanal display code but I wanted to keep it incase I wanted to add that in later.

input.php
Code:
<?php
include_once('db.inc');
// Do not alter these global variables
$curTag = "";
$serverName = "";
$planetName = "";
$resourceData = array();

// Customize this method to update your database. You will probably want to replace the
// sample code here with a series of SQL statements customized for your database.
function handleResource() {
global $resourceData, $serverName, $planetName, $current_server;
// Remove this "if" statment if you want to process all servers, or change
// it to match the single server you are interested in.
if ((!$current_server)){
echo "No server Info was passed to me";
exit();
}
if ($serverName == $current_server) {
$availTime = $resourceData["AVAILABLE_TIMESTAMP"];

//check if resource Type exists in db if not INSERT it
$slashtype = $resourceData['TYPE'];
$slashtype = addslashes($slashtype);
$sql_resource_type_check = mysql_query("SELECT rtype_name FROM rtype WHERE rtype_name='$slashtype'");
$resource_type_check = mysql_num_rows($sql_resource_type_check);
if($resource_type_check <1) {
$sql_type = mysql_query("
INSERT INTO rtype (rtype_name)
VALUES('$slashtype') ")
or die (mysql_error());
}
//check if rname already exists in db if not INSERT it along with the rtid
//get rtid
$query_type = "SELECT rtid FROM rtype WHERE rtype_name = '$slashtype'";
$type = mysql_query($query_type) or die(mysql_error());
$rs_type = mysql_result($type, 0, 0);
$rtid = $rs_type;

//check if rname is in db or not
$slashname = $resourceData['NAME'];
$slashname = addslashes($slashname);
$sql_resource_name_check = mysql_query("SELECT rname_name FROM rname WHERE rname_name='$slashname'");
$resource_name_check = mysql_num_rows($sql_resource_name_check);
//if not insert it
if($resource_name_check <1) {
$sql_name = mysql_query("
INSERT INTO rname (rtid, rname_name, cd, cr, dr, er, fl, hr, ma, oq, pe, sr, ut)
VALUES('$rtid', '$slashname', '$resourceData[CD]', '$resourceData[CR]', '$resourceData[DR]', '$resourceData[ER]', '$resourceData[FL]', '$resourceData[HR]', '$resourceData[MA]', '$resourceData[OQ]', '$resourceData[PE]', '$resourceData[SR]', '$resourceData[UT]') ")
or die (mysql_error());


/*
$output = sprintf("<p>%s %s <b>%s </b>- %s : CD = %d, CR = %d, DR = %d, ER = %d, FL = %d, HR = %d, " .
"MA = %d, OQ = %d, PE = %d, SR = %d, UT = %d, Available on: %s<BR>",
$serverName,
$planetName,
$resourceData["NAME"],
$resourceData["TYPE"],
$resourceData["CD"],
$resourceData["CR"],
$resourceData["DR"],
$resourceData["ER"],
$resourceData["FL"],
$resourceData["HR"],
$resourceData["MA"],
$resourceData["OQ"],
$resourceData["PE"],
$resourceData["SR"],
$resourceData["UT"],
date('Y-m-d', $availTime));

echo($output); */
}
}
}




// Clears the $resourceData array.
// DO NOT MODIFY THIS FUNCTION
function clearResourceData() {
global $resourceData;
$resourceData["NAME"] = "";
$resourceData["CD"] = 0;
$resourceData["CR"] = 0;
$resourceData["DR"] = 0;
$resourceData["ER"] = 0;
$resourceData["FL"] = 0;
$resourceData["HR"] = 0;
$resourceData["MA"] = 0;
$resourceData["OQ"] = 0;
$resourceData["PE"] = 0;
$resourceData["SR"] = 0;
$resourceData["UT"] = 0;
}

// DO NOT MODIFY
function startElement($parser, $elementName, $elementAttrs) {
global $serverName, $planetName, $curTag;
$curTag = $elementName;

switch($elementName) {
case "SERVER":
$serverName = $elementAttrs["NAME"];
break;

case "PLANET":
$planetName = $elementAttrs["NAME"];
break;
}
}

// We have received an ending tag.
// DO NOT MODIFY
function endElement($parser, $elementName) {
switch($elementName) {
case "RESOURCE":
// We have gathered all of the information for this resource.
handleResource();
clearResourceData();
break;
}
}


// This is regular character data. We are only interested in it if
// the current tag is RESOURCE.
// DO NOT MODIFY
function characterData($parser, $data) {
global $resourceData, $curTag;

$data = trim($data);

if(strlen($data) > 0 && $data != " "):
$resourceData[$curTag] = $data;
endif;
}


// Connect to the SWGCraft.com site, open the resource file and parse the info.
// DO NOT MODIFY
function parseResourceXML() {
// Start of the main body of code
$parser = xml_parser_create();
xml_set_element_handler($parser, 'startElement', 'endElement');
xml_set_character_data_handler($parser, 'characterData');
//$fp = gzopen(' http://www.swgcraft.com/sendfile.php?file=currentresources_chilastra.xml.gz', 'r') or die("<p>Cannot open the XML source file");
//$fp = gzopen('http://www.swgcraft.com/static/resources/currentresources.xml.gz', 'r') or die("<p>Cannot open the XML source file");
$fp = gzopen('currentresources.xml.gz', 'r') or die("<p>Cannot open the XML source file");
while($data = gzread($fp, 4096)) {
$result = xml_parse($parser, $data, gzeof($fp)); // or die("XML ERROR");
if(!$result) {
$errMsg = xml_error_string(xml_get_error_code($parser));
$lineNum = xml_get_current_line_number($parser);
$colNum = xml_get_current_column_number($parser);
echo($errMsg . " on line blah blah $lineNum, column $colNum\n");
die("Error parsing the file.");
}
}

// Close the XML file
gzclose($fp);
xml_parser_free($parser);
}
?>
<?php
//ok functions are done. Lets mae it happen
//get value from form
$server = $_POST['sid'];

//show form if POST not found
if ((!$server)) {
include 'server.html';
exit();
}
//form has been shown and a value has been returned
//translate sid to $current_server
//get proper results based in sid from $_POST
$sql_server_result="SELECT server FROM server WHERE sid='$server'";
$server_result =mysql_query($sql_server_result) or die("Did not select server name");
$rs_server_result = mysql_result($server_result, 0, 0);
$current_server = $rs_server_result;

//get total number of entries currently in db.
// both for type and name
$sql_totals="SELECT * FROM resource_totals";
$totals =mysql_query($sql_totals) or die("did not get resource totals");
$rs_totals = mysql_fetch_assoc($totals);
$totalid = $rs_totals['rtotalid'];
$total_type = $rs_totals['type_total'];
$total_name = $rs_totals['name_total'];

echo "You are currently updating $current_server <br>";
echo "There are currently: <br> types: $total_type <br> Names: $total_name. <br> in the database.";


//pushing the big red button!!! mmmwwwwuuuuaahahahahahaa
parseResourceXML();

//get new totals.
//types
$sql_new_types="SELECT * FROM rtype";
$new_types =mysql_query($sql_new_types) or die("did not get rtypes");
$rs_new_types = mysql_fetch_assoc($new_types);
$new_total_types = mysql_num_rows($new_types);
$final_types = ($new_total_types - $total_type);
//names
$sql_new_names="SELECT * FROM rname";
$new_names =mysql_query($sql_new_names) or die("did not get rname");
$rs_new_names = mysql_fetch_assoc($new_names);
$new_total_names = mysql_num_rows($new_names);

$final_names = ($new_total_names - $total_name);
echo "<p> $current_sever is completed. You have added:<br>
         types: $final_types <br> Names: $final_names <br> in the database.";
echo "<p> <a href=input.php>BACK</a>";
//update totals into db
$sql = "UPDATE resource_totals SET type_total = '$final_types', name_total = '$final_names' WHERE rtotalid = '$totalid'";
$result = mysql_query ($sql) or die("Database Query Failed to update totals.<br>");





/*
<HTML>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<!-- Modify to use your stylesheet, if you use one at all -->
<LINK REL="stylesheet" TYPE="text/css" HREF="stylesheet.css">
<title>Update Resources</title>
</head>
<BODY>
<p>Updating resource information from SWGCraft.com</p>

<?php
parseResourceXML();


<p>Resources have been updated!</p>
</BODY>*/

?>


server.html
Code:
<?php
include_once('db.inc');

$query_server = "SELECT * FROM server ORDER BY server";
$server = mysql_query($query_server) or die(mysql_error());
$rs_server = mysql_fetch_assoc($server);
$totalRows_rs_server = mysql_num_rows($server);
?>

<form name="server" method="post" action="input.php">
  <table width="400">
    <tr>
      <td valign="top"><strong>Server Name</strong></td>
      <td>&nbsp;</td>
      <td valign="top"><strong>Lasted Updated</strong>
       </td>
  </tr>
<?php
do {
?>
<tr>
      <td><label><?php echo $rs_server['server']; ?></label></td>
      <td  align=right><input type="radio" name="sid" value=<?php echo $rs_server['sid'];?>></td>
     <td align=left> <span class="verysmall"><font color=red>place lasted updated here</font></span></td>
    </tr>   
<?php
}

while ($rs_server = mysql_fetch_assoc($server));
  $rows = mysql_num_rows($server);
  if($rows > 0) {
      mysql_data_seek($server, 0);
     $rs_server = mysql_fetch_assoc($server);
  }
  ?>
<tr>
      <td>&nbsp;</td>
      <td  align=center><input type=submit name=Submit value=Submit></td>
    </tr>
  </table>
</form>
<?php
mysql_free_result($server);
?>

_________________
Moge


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 03, 2004 2:48 am 
Offline
User avatar

Joined: Thu Jan 16, 2003 8:15 pm
Posts: 734
Location: Norway, Bergen
mmmmm ice cream

_________________
Avian Technology & Trade

Wermas Tanares
-Master Weaponsmith-
-Master Pistoleer-

TC Char
Nawa


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group